[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:37 dist-test-slave-4tdj krb5kdc[27391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:37 dist-test-slave-4tdj krb5kdc[27391](info): set up 2 sockets
Aug 07 22:42:37 dist-test-slave-4tdj krb5kdc[27391](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 22:42:38.151355 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:38.269845 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:33659
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:33659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:38.376309 27412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:38.376538 27412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:38.376601 27412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:38.380785 27412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:38.380838 27412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:38.380863 27412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:38.380884 27412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:38.380913 27412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:38.385916 27412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:33659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:33659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:38.386893 27412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:38.387830 27412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:38.395805 27427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:38.395988 27412 server_base.cc:1034] running on GCE node
W20260807 22:42:38.395834 27430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:38.395834 27426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:38.396716 27412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:38.397707 27412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:38.398866 27412 hybrid_clock.cc:648] HybridClock initialized: now 1786142558398839 us; error 39 us; skew 500 ppm
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:38.401434 27412 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:38.402252 27412 webserver.cc:460] Webserver started at http://127.26.189.254:42597/ using document root <none> and password file <none>
I20260807 22:42:38.402789 27412 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:38.402858 27412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:38.403053 27412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:38.405113 27412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "3046bc1a5c734275aa747d8c8f2d3532"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "96de8b2e24920d148400bb4da1aac06a"
server_key_iv: "d26df7e53af0b7ea9db895be64b36d49"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.405671 27412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "3046bc1a5c734275aa747d8c8f2d3532"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "96de8b2e24920d148400bb4da1aac06a"
server_key_iv: "d26df7e53af0b7ea9db895be64b36d49"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.409265 27412 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:42:38.411612 27439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.412770 27412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:38.412882 27412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "3046bc1a5c734275aa747d8c8f2d3532"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "96de8b2e24920d148400bb4da1aac06a"
server_key_iv: "d26df7e53af0b7ea9db895be64b36d49"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.412972 27412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:38.434451 27412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:38.437126 27412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:38.437297 27412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:38.443908 27412 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:33659
I20260807 22:42:38.443931 27519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:33659 every 8 connection(s)
I20260807 22:42:38.444777 27412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:38.445678 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27412
I20260807 22:42:38.445832 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:38.447147 27383 external_mini_cluster.cc:1428] Setting key bcf4a1040eb8273eae2a91678b80ea40
I20260807 22:42:38.447536 27520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:38.453680 27520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532: Bootstrap starting.
I20260807 22:42:38.456125 27520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:38.457015 27520 log.cc:824] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:38.459143 27520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532: No bootstrap required, opened a new log
I20260807 22:42:38.463194 27520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046bc1a5c734275aa747d8c8f2d3532" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33659 } }
I20260807 22:42:38.463451 27520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:38.463513 27520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3046bc1a5c734275aa747d8c8f2d3532, State: Initialized, Role: FOLLOWER
I20260807 22:42:38.463907 27520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046bc1a5c734275aa747d8c8f2d3532" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33659 } }
I20260807 22:42:38.464031 27520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:38.464097 27520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:38.464190 27520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:38.465034 27520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046bc1a5c734275aa747d8c8f2d3532" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33659 } }
I20260807 22:42:38.465416 27520 leader_election.cc:302] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3046bc1a5c734275aa747d8c8f2d3532; no voters:
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:38.465785 27520 leader_election.cc:288] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:38.465843 27530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:38.466066 27530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [term 1 LEADER]: Becoming Leader. State: Replica: 3046bc1a5c734275aa747d8c8f2d3532, State: Running, Role: LEADER
I20260807 22:42:38.466466 27530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046bc1a5c734275aa747d8c8f2d3532" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33659 } }
I20260807 22:42:38.467064 27520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:38.468169 27532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3046bc1a5c734275aa747d8c8f2d3532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046bc1a5c734275aa747d8c8f2d3532" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33659 } } }
I20260807 22:42:38.468292 27532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:38.468710 27542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:38.468737 27533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3046bc1a5c734275aa747d8c8f2d3532. Latest consensus state: current_term: 1 leader_uuid: "3046bc1a5c734275aa747d8c8f2d3532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046bc1a5c734275aa747d8c8f2d3532" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33659 } } }
I20260807 22:42:38.468853 27533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:38.471953 27542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:38.472736 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:38.450189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.26.189.254:33659)
0807 22:42:38.450657 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:38.450675 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:38.451793 (+ 1118us) server_negotiation.cc:407] Connection header received
0807 22:42:38.455659 (+ 3866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:38.455673 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:38.455957 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:38.456229 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:38.458947 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.459912 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.461544 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.461996 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.462122 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:38.466518 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:38.466524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:38.466539 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:38.466570 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:38.468733 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:38.469425 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:38.469431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:38.469437 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:38.469595 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:38.470000 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:38.470004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:38.470006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:38.470522 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:38.470739 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:38.471563 (+ 824us) server_negotiation.cc:299] Negotiation successful
0807 22:42:38.471847 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":130,"threads_started":1}
I20260807 22:42:38.477638 27542 catalog_manager.cc:1347] Generated new cluster ID: fdb6be0635924a70ad0e5724a181a01b
I20260807 22:42:38.477712 27542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:38.493012 27542 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:38.493952 27542 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:38.499238 27542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3046bc1a5c734275aa747d8c8f2d3532: Generated new TSK 0
I20260807 22:42:38.499855 27542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:38.530649 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:33659
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:38.640202 27560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:38.640441 27560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:38.640498 27560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:38.644423 27560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:38.644476 27560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:38.644551 27560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:38.649722 27560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:33659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:38.650743 27560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:38.651710 27560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:38.661001 27569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:38.661041 27568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:38.661062 27560 server_base.cc:1034] running on GCE node
W20260807 22:42:38.661010 27571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:38.661610 27560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:38.662227 27560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:38.663378 27560 hybrid_clock.cc:648] HybridClock initialized: now 1786142558663361 us; error 28 us; skew 500 ppm
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:38.665927 27560 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:38.666759 27560 webserver.cc:460] Webserver started at http://127.26.189.193:37353/ using document root <none> and password file <none>
I20260807 22:42:38.667286 27560 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:38.667356 27560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:38.667563 27560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:38.669528 27560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "465be1836c4d4f85898e75ab81d7a6c0"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "5e5c4e1b9d323240d87cdf018b859c0c"
server_key_iv: "49358736781e59e3037f60ebfab6588c"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.670060 27560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "465be1836c4d4f85898e75ab81d7a6c0"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "5e5c4e1b9d323240d87cdf018b859c0c"
server_key_iv: "49358736781e59e3037f60ebfab6588c"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.673517 27560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:42:38.675781 27579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.676919 27560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:38.677023 27560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "465be1836c4d4f85898e75ab81d7a6c0"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "5e5c4e1b9d323240d87cdf018b859c0c"
server_key_iv: "49358736781e59e3037f60ebfab6588c"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.677124 27560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:38.696245 27560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:38.700145 27560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:38.700301 27560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:38.700834 27560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:38.701615 27560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:38.701663 27560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.701700 27560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:38.701720 27560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.710399 27560 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:34197
I20260807 22:42:38.710435 27748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:34197 every 8 connection(s)
I20260807 22:42:38.711275 27560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:38.716801 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27560
I20260807 22:42:38.716888 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:38.717115 27383 external_mini_cluster.cc:1428] Setting key 74766431b718186af256f52ba1afb626
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:38.724653 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:38.713149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:48813 (local address 127.26.189.254:33659)
0807 22:42:38.713278 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:38.713282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:38.714282 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 22:42:38.715184 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:38.715187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:38.715244 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:38.715314 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:38.716512 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.717121 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.718245 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.718530 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.718625 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:38.721194 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:38.721198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:38.721201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:38.721230 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:38.722687 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:38.723153 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:38.723156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:38.723158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:38.723251 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:38.723586 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:38.723589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:38.723591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:38.723810 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:38.723909 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:38.724350 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 22:42:38.724481 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:42:38.725373 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:38.713496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:33659 (local address 127.26.189.193:48813)
0807 22:42:38.714135 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:38.714177 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:38.714983 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:38.715501 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:38.715508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:38.715869 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:38.716353 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:38.716367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.717263 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.717266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:38.718126 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:38.718136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.718689 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.718693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:38.718843 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:38.719392 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:38.719407 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:38.721028 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:38.722790 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:38.722794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:38.722807 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:38.723049 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:38.723337 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:38.723340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:38.723342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:38.723500 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:38.723896 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:38.723901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:38.724124 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 22:42:38.724271 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:42:38.724509 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":113,"threads_started":1}
I20260807 22:42:38.726557 27752 heartbeater.cc:347] Connected to a master server at 127.26.189.254:33659
I20260807 22:42:38.726780 27752 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:42:38.727246 27752 heartbeater.cc:511] Master 127.26.189.254:33659 requested a full tablet report, sending...
I20260807 22:42:38.728690 27461 ts_manager.cc:194] Registered new tserver with Master: 465be1836c4d4f85898e75ab81d7a6c0 (127.26.189.193:34197)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:42:38.729832 27461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:48813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:38.743163 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:33659
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:38.852020 27761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:38.852296 27761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:38.852362 27761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:38.856293 27761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:38.856348 27761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:38.856424 27761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:38.861609 27761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:33659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:38.862588 27761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:38.863530 27761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:38.872665 27771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:38.872822 27761 server_base.cc:1034] running on GCE node
W20260807 22:42:38.872678 27777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:38.872710 27772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:38.873298 27761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:38.873890 27761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:38.875033 27761 hybrid_clock.cc:648] HybridClock initialized: now 1786142558875012 us; error 38 us; skew 500 ppm
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:38.877573 27761 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:38.878391 27761 webserver.cc:460] Webserver started at http://127.26.189.194:35017/ using document root <none> and password file <none>
I20260807 22:42:38.878901 27761 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:38.878960 27761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:38.879161 27761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:38.881193 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "e083502fe0a84f19b1171f0a9bc2881c"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "1988126a0c2222f0d59adf5a05ad7350"
server_key_iv: "62a6fb51c7f13ef5afe56a8780b5dc05"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.881721 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "e083502fe0a84f19b1171f0a9bc2881c"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "1988126a0c2222f0d59adf5a05ad7350"
server_key_iv: "62a6fb51c7f13ef5afe56a8780b5dc05"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.885313 27761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:42:38.887637 27788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.888764 27761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:38.888871 27761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "e083502fe0a84f19b1171f0a9bc2881c"
format_stamp: "Formatted at 2026-08-07 22:42:38 on dist-test-slave-4tdj"
server_key: "1988126a0c2222f0d59adf5a05ad7350"
server_key_iv: "62a6fb51c7f13ef5afe56a8780b5dc05"
server_key_version: "encryptionkey@0"
I20260807 22:42:38.888969 27761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:38.906070 27761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:38.908732 27761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:38.908910 27761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:38.909513 27761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:38.910353 27761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:38.910400 27761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.910451 27761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:38.910478 27761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:38.919045 27761 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:43725
I20260807 22:42:38.919098 27980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:43725 every 8 connection(s)
I20260807 22:42:38.919916 27761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:42:38 dist-test-slave-4tdj krb5kdc[27391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:38.929153 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27761
I20260807 22:42:38.929273 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:38.929555 27383 external_mini_cluster.cc:1428] Setting key 33a23840260808daffb0f5702f87597a
I20260807 22:42:38.932528 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:38.921734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:59273 (local address 127.26.189.254:33659)
0807 22:42:38.921857 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:38.921861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:38.922872 (+ 1011us) server_negotiation.cc:407] Connection header received
0807 22:42:38.923739 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:38.923743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:38.923800 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:38.923871 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:38.924977 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.925518 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.926667 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.926887 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.926977 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:38.929197 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:38.929201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:38.929204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:38.929233 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:38.930666 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:38.931161 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:38.931165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:38.931166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:38.931261 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:38.931545 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:38.931549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:38.931550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:38.931751 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:38.931850 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:38.932258 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 22:42:38.932375 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:42:38.933250 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:38.922081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:33659 (local address 127.26.189.194:59273)
0807 22:42:38.922736 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:38.922776 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:38.923531 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:38.924045 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:38.924051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:38.924387 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:38.924833 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:38.924846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.925645 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.925648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:38.926545 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:38.926556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:38.927025 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:38.927029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:38.927183 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:38.927657 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:38.927673 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:38.929037 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:38.930775 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:38.930781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:38.930794 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:38.931066 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:38.931343 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:38.931345 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:38.931347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:38.931465 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:38.931835 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:38.931840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:38.932050 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 22:42:38.932195 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 22:42:38.932423 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":114,"threads_started":1}
I20260807 22:42:38.934321 27981 heartbeater.cc:347] Connected to a master server at 127.26.189.254:33659
I20260807 22:42:38.934551 27981 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:38.935062 27981 heartbeater.cc:511] Master 127.26.189.254:33659 requested a full tablet report, sending...
I20260807 22:42:38.936127 27461 ts_manager.cc:194] Registered new tserver with Master: e083502fe0a84f19b1171f0a9bc2881c (127.26.189.194:43725)
I20260807 22:42:38.936671 27461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:59273
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:38.956688 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:33659
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:39.068230 27989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:39.068472 27989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:39.068535 27989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:39.072540 27989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:39.072598 27989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:39.072676 27989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:39.077724 27989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:33659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:39.078754 27989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:39.079718 27989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:39.089151 27995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:39.089196 27998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:39.089162 27996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:39.089274 27989 server_base.cc:1034] running on GCE node
I20260807 22:42:39.089730 27989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:39.090351 27989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:39.091518 27989 hybrid_clock.cc:648] HybridClock initialized: now 1786142559091500 us; error 31 us; skew 500 ppm
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142559, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:39.094094 27989 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:39.094933 27989 webserver.cc:460] Webserver started at http://127.26.189.195:36489/ using document root <none> and password file <none>
I20260807 22:42:39.095445 27989 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:39.095512 27989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:39.095712 27989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:39.097644 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "425389ecc6754908b474817f42e60f7b"
format_stamp: "Formatted at 2026-08-07 22:42:39 on dist-test-slave-4tdj"
server_key: "af876a3eab1e4e8ddc93ae09104b9f65"
server_key_iv: "a3f47f40b3adcb2069b784fc97b3c77a"
server_key_version: "encryptionkey@0"
I20260807 22:42:39.098170 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "425389ecc6754908b474817f42e60f7b"
format_stamp: "Formatted at 2026-08-07 22:42:39 on dist-test-slave-4tdj"
server_key: "af876a3eab1e4e8ddc93ae09104b9f65"
server_key_iv: "a3f47f40b3adcb2069b784fc97b3c77a"
server_key_version: "encryptionkey@0"
I20260807 22:42:39.101588 27989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260807 22:42:39.103953 28007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:39.105093 27989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:39.105197 27989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "425389ecc6754908b474817f42e60f7b"
format_stamp: "Formatted at 2026-08-07 22:42:39 on dist-test-slave-4tdj"
server_key: "af876a3eab1e4e8ddc93ae09104b9f65"
server_key_iv: "a3f47f40b3adcb2069b784fc97b3c77a"
server_key_version: "encryptionkey@0"
I20260807 22:42:39.105298 27989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:39.120416 27989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:39.123358 27989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:39.123539 27989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:39.124099 27989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:39.124908 27989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:39.124966 27989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:39.125017 27989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:39.125047 27989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:39.133193 27989 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:37987
I20260807 22:42:39.133252 28173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:37987 every 8 connection(s)
I20260807 22:42:39.134039 27989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:39.141177 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27989
I20260807 22:42:39.141278 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:39.141505 27383 external_mini_cluster.cc:1428] Setting key 85ad4014813464a7f6b984233a61b54f
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[27391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142559, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:39.146687 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:39.135826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:57297 (local address 127.26.189.254:33659)
0807 22:42:39.135945 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:39.135949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:39.136910 (+ 961us) server_negotiation.cc:407] Connection header received
0807 22:42:39.137799 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:39.137802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:39.137863 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:39.137933 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:39.139116 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.139646 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.140750 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.140982 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.141087 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:39.143443 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:39.143446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:39.143448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:39.143472 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:39.144777 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:39.145266 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:39.145269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:39.145270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:39.145346 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:39.145654 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:39.145658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:39.145659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:39.145832 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:39.145916 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:39.146403 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 22:42:39.146521 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:42:39.147478 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:39.136149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:33659 (local address 127.26.189.195:57297)
0807 22:42:39.136763 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:39.136803 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:39.137593 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:39.138113 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:39.138120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:39.138514 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:39.138972 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:39.138986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.139766 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.139769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:39.140628 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:39.140639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.141137 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.141141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:39.141311 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:39.141866 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:39.141882 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:39.143282 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:39.144885 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:39.144891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:39.144910 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:39.145158 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:39.145428 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:39.145431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:39.145433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:39.145566 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:39.145931 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:39.145937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:39.146173 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 22:42:39.146353 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 22:42:39.146656 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":105,"threads_started":1}
I20260807 22:42:39.148424 28175 heartbeater.cc:347] Connected to a master server at 127.26.189.254:33659
I20260807 22:42:39.148638 28175 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:39.149080 28175 heartbeater.cc:511] Master 127.26.189.254:33659 requested a full tablet report, sending...
I20260807 22:42:39.150008 27461 ts_manager.cc:194] Registered new tserver with Master: 425389ecc6754908b474817f42e60f7b (127.26.189.195:37987)
I20260807 22:42:39.150630 27461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:57297
I20260807 22:42:39.155360 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142559, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[27391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142559, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:39.180868 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:39.171915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.26.189.193:34197)
0807 22:42:39.172282 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:39.172292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:39.172316 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:42:39.172364 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:39.172370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:39.172543 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:39.172698 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:39.173285 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.174040 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.174921 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.175284 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.175368 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:39.177714 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:39.177723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:39.177736 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:39.177759 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:39.179289 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:39.179694 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:39.179699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:39.179700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:39.179773 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:39.179967 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:39.179969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:39.179970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:39.180194 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:39.180356 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:39.180543 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 22:42:39.180676 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":159,"threads_started":1}
W20260807 22:42:39.181845 27623 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42728
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[27391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[27391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:39.206068 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:39.198513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.26.189.193:34197)
0807 22:42:39.198658 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:39.198662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:39.198800 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:42:39.198926 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:39.198930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:39.198973 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:39.199037 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:39.199750 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.200258 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.201075 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:39.201321 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:39.201408 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:39.203602 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:39.203605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:39.203607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:39.203630 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:39.204860 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:39.205176 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:39.205178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:39.205179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:39.205256 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:39.205431 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:39.205433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:39.205434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:39.205605 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:39.205698 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:39.205859 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 22:42:39.205950 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:42:39.207012 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27560
I20260807 22:42:39.218546 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27761
I20260807 22:42:39.233278 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27989
I20260807 22:42:39.243026 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27412
2026-08-07T22:42:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2255 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[28191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[28191](info): set up 2 sockets
Aug 07 22:42:39 dist-test-slave-4tdj krb5kdc[28191](info): commencing operation
krb5kdc: starting...
W20260807 22:42:40.375067 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:40 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:40.491178 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:44523
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:40.597754 28208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:40.597997 28208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:40.598035 28208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:40.601899 28208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:40.601951 28208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:40.601969 28208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:40.601984 28208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:40.601997 28208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:40.607069 28208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:40.608153 28208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:40.609095 28208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:40.617259 28215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:40.617244 28218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:40.617225 28216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:40.617314 28208 server_base.cc:1034] running on GCE node
I20260807 22:42:40.617955 28208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:40.619016 28208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:40.620168 28208 hybrid_clock.cc:648] HybridClock initialized: now 1786142560620151 us; error 32 us; skew 500 ppm
Aug 07 22:42:40 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142560, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:40.622726 28208 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:40.623553 28208 webserver.cc:460] Webserver started at http://127.26.189.254:41793/ using document root <none> and password file <none>
I20260807 22:42:40.624063 28208 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:40.624136 28208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:40.624329 28208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:40.626286 28208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "0f067569f29e4b32adc37b435e9b6377"
format_stamp: "Formatted at 2026-08-07 22:42:40 on dist-test-slave-4tdj"
server_key: "b452dc357439a3128904162badd037be"
server_key_iv: "c779aaf941c07c83d9ed290b249c3bb1"
server_key_version: "encryptionkey@0"
I20260807 22:42:40.626847 28208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "0f067569f29e4b32adc37b435e9b6377"
format_stamp: "Formatted at 2026-08-07 22:42:40 on dist-test-slave-4tdj"
server_key: "b452dc357439a3128904162badd037be"
server_key_iv: "c779aaf941c07c83d9ed290b249c3bb1"
server_key_version: "encryptionkey@0"
I20260807 22:42:40.630241 28208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:40.632431 28232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:40.633638 28208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 22:42:40.633757 28208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "0f067569f29e4b32adc37b435e9b6377"
format_stamp: "Formatted at 2026-08-07 22:42:40 on dist-test-slave-4tdj"
server_key: "b452dc357439a3128904162badd037be"
server_key_iv: "c779aaf941c07c83d9ed290b249c3bb1"
server_key_version: "encryptionkey@0"
I20260807 22:42:40.633848 28208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:40.673188 28208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:40.675973 28208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:40.676165 28208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:40.682875 28208 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:44523
I20260807 22:42:40.682899 28317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:44523 every 8 connection(s)
I20260807 22:42:40.683768 28208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:40.686461 28318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:40.686739 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28208
I20260807 22:42:40.686854 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:40.687150 27383 external_mini_cluster.cc:1428] Setting key 9e78f61f5e138938a32e3c0187fa1d94
I20260807 22:42:40.693069 28318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377: Bootstrap starting.
I20260807 22:42:40.695364 28318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:40.696043 28318 log.cc:824] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:40.698041 28318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377: No bootstrap required, opened a new log
Aug 07 22:42:40 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:40.701058 28318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f067569f29e4b32adc37b435e9b6377" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44523 } }
I20260807 22:42:40.701272 28318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:40.701340 28318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f067569f29e4b32adc37b435e9b6377, State: Initialized, Role: FOLLOWER
I20260807 22:42:40.701817 28318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f067569f29e4b32adc37b435e9b6377" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44523 } }
I20260807 22:42:40.701982 28318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:40.702050 28318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:40.702149 28318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:40.703020 28318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f067569f29e4b32adc37b435e9b6377" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44523 } }
I20260807 22:42:40.703392 28318 leader_election.cc:302] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f067569f29e4b32adc37b435e9b6377; no voters:
I20260807 22:42:40.703763 28318 leader_election.cc:288] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:40.703940 28324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:40.704169 28324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [term 1 LEADER]: Becoming Leader. State: Replica: 0f067569f29e4b32adc37b435e9b6377, State: Running, Role: LEADER
I20260807 22:42:40.704478 28324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f067569f29e4b32adc37b435e9b6377" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44523 } }
I20260807 22:42:40.704834 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:40.688425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.26.189.254:44523)
0807 22:42:40.688819 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:40.688835 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:40.688868 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:42:40.689629 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:40.689650 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:40.689954 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:40.690213 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:40.691094 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:40.692046 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:40.693165 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:40.693635 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:40.693745 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:40.699376 (+ 5631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:40.699381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:40.699393 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:40.699418 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:40.701453 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:40.701914 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:40.701919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:40.701923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:40.702058 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:40.702375 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:40.702378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:40.702380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:40.702765 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:40.702939 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:40.703238 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 22:42:40.703681 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
I20260807 22:42:40.705080 28318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:40.706770 28324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f067569f29e4b32adc37b435e9b6377. Latest consensus state: current_term: 1 leader_uuid: "0f067569f29e4b32adc37b435e9b6377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f067569f29e4b32adc37b435e9b6377" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44523 } } }
I20260807 22:42:40.706764 28325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f067569f29e4b32adc37b435e9b6377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f067569f29e4b32adc37b435e9b6377" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44523 } } }
I20260807 22:42:40.707010 28324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:40.707304 28325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:40.707818 28334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:40.711079 28334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:40.716503 28334 catalog_manager.cc:1347] Generated new cluster ID: c92164c6094341e4ba7f7178865c2889
I20260807 22:42:40.716576 28334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:40.737388 28334 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:40.738673 28334 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:40.745793 28334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f067569f29e4b32adc37b435e9b6377: Generated new TSK 0
I20260807 22:42:40.746634 28334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:40.781101 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44523
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:40.890615 28357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:40.890866 28357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:40.890928 28357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:40.894896 28357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:40.894948 28357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:40.895026 28357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:40.900177 28357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44523
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:40.901161 28357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:40.902150 28357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:40.911448 28364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:40.911576 28367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:40.911485 28357 server_base.cc:1034] running on GCE node
W20260807 22:42:40.911460 28363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:40.912067 28357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:40.912688 28357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:40.913843 28357 hybrid_clock.cc:648] HybridClock initialized: now 1786142560913828 us; error 28 us; skew 500 ppm
Aug 07 22:42:40 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142560, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:40.916388 28357 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:40.917203 28357 webserver.cc:460] Webserver started at http://127.26.189.193:35639/ using document root <none> and password file <none>
I20260807 22:42:40.917701 28357 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:40.917764 28357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:40.917961 28357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:40.919919 28357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "415590a7b83642488ebfe50f419abc35"
format_stamp: "Formatted at 2026-08-07 22:42:40 on dist-test-slave-4tdj"
server_key: "b42d3fa380c01cf2b08604bafeb5cfd3"
server_key_iv: "5432e88f4212efe2a934a73763b071f5"
server_key_version: "encryptionkey@0"
I20260807 22:42:40.920440 28357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "415590a7b83642488ebfe50f419abc35"
format_stamp: "Formatted at 2026-08-07 22:42:40 on dist-test-slave-4tdj"
server_key: "b42d3fa380c01cf2b08604bafeb5cfd3"
server_key_iv: "5432e88f4212efe2a934a73763b071f5"
server_key_version: "encryptionkey@0"
I20260807 22:42:40.923904 28357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 22:42:40.926223 28379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:40.927412 28357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:40.927529 28357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "415590a7b83642488ebfe50f419abc35"
format_stamp: "Formatted at 2026-08-07 22:42:40 on dist-test-slave-4tdj"
server_key: "b42d3fa380c01cf2b08604bafeb5cfd3"
server_key_iv: "5432e88f4212efe2a934a73763b071f5"
server_key_version: "encryptionkey@0"
I20260807 22:42:40.927632 28357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:40.944635 28357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:40.947295 28357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:40.947443 28357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:40.947950 28357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:40.948720 28357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:40.948766 28357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:40.948805 28357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:40.948825 28357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:40.957590 28357 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:45589
I20260807 22:42:40.957638 28550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:45589 every 8 connection(s)
I20260807 22:42:40.958508 28357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:40.967749 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28357
Aug 07 22:42:40 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142560, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:40.967876 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:40.968144 27383 external_mini_cluster.cc:1428] Setting key 9e071589aaea36d89aac2e90d49fe5f9
I20260807 22:42:40.971822 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:40.960257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:34991 (local address 127.26.189.254:44523)
0807 22:42:40.960399 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:40.960403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:40.961246 (+ 843us) server_negotiation.cc:407] Connection header received
0807 22:42:40.962581 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:40.962585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:40.962644 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:40.962718 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:40.963899 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:40.964448 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:40.965609 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:40.965845 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:40.965933 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:40.968475 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:40.968479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:40.968482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:40.968512 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:40.969866 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:40.970359 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:40.970363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:40.970365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:40.970445 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:40.970773 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:40.970776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:40.970777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:40.970950 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:40.971036 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:40.971528 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 22:42:40.971658 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:42:40.972632 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:40.960624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44523 (local address 127.26.189.193:34991)
0807 22:42:40.961092 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:40.961137 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:40.962359 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:40.962899 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:40.962906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:40.963261 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:40.963752 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:40.963765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:40.964576 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:40.964579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:40.965478 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:40.965488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:40.965989 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:40.965993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:40.966160 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:40.966734 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:40.966751 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:40.968302 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:40.969978 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:40.969985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:40.970001 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:40.970252 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:40.970528 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:40.970532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:40.970534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:40.970676 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:40.971049 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:40.971056 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:40.971293 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 22:42:40.971454 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 22:42:40.971796 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260807 22:42:40.973834 28552 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44523
I20260807 22:42:40.974116 28552 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:40.974747 28552 heartbeater.cc:511] Master 127.26.189.254:44523 requested a full tablet report, sending...
I20260807 22:42:40.976485 28263 ts_manager.cc:194] Registered new tserver with Master: 415590a7b83642488ebfe50f419abc35 (127.26.189.193:45589)
I20260807 22:42:40.978044 28263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:34991
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:40.995157 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44523
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:41.112769 28560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:41.113102 28560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:41.113170 28560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:41.118230 28560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:41.118326 28560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:41.118423 28560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:41.125653 28560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44523
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:41.126916 28560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:41.127977 28560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:41.137898 28567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:41.137982 28566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:41.137987 28571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:41.138408 28560 server_base.cc:1034] running on GCE node
I20260807 22:42:41.138854 28560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:41.139614 28560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:41.140798 28560 hybrid_clock.cc:648] HybridClock initialized: now 1786142561140770 us; error 45 us; skew 500 ppm
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:41.143572 28560 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:41.144363 28560 webserver.cc:460] Webserver started at http://127.26.189.194:39387/ using document root <none> and password file <none>
I20260807 22:42:41.144865 28560 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:41.144920 28560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:41.145087 28560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:41.147073 28560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "6234d7309a1b461194ec7b1866b87aa8"
format_stamp: "Formatted at 2026-08-07 22:42:41 on dist-test-slave-4tdj"
server_key: "6e41d1227eaa4aacb945c07aa1325c6a"
server_key_iv: "27e5f91386558cfaa7909846c227560c"
server_key_version: "encryptionkey@0"
I20260807 22:42:41.147574 28560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "6234d7309a1b461194ec7b1866b87aa8"
format_stamp: "Formatted at 2026-08-07 22:42:41 on dist-test-slave-4tdj"
server_key: "6e41d1227eaa4aacb945c07aa1325c6a"
server_key_iv: "27e5f91386558cfaa7909846c227560c"
server_key_version: "encryptionkey@0"
I20260807 22:42:41.151036 28560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:42:41.153258 28580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:41.154464 28560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 22:42:41.154560 28560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "6234d7309a1b461194ec7b1866b87aa8"
format_stamp: "Formatted at 2026-08-07 22:42:41 on dist-test-slave-4tdj"
server_key: "6e41d1227eaa4aacb945c07aa1325c6a"
server_key_iv: "27e5f91386558cfaa7909846c227560c"
server_key_version: "encryptionkey@0"
I20260807 22:42:41.154663 28560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:41.164569 28560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:41.167469 28560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:41.167635 28560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:41.168188 28560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:41.168952 28560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:41.168999 28560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:41.169049 28560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:41.169078 28560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:41.177333 28560 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:40057
I20260807 22:42:41.177386 28751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:40057 every 8 connection(s)
I20260807 22:42:41.178175 28560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:41.181565 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28560
I20260807 22:42:41.181665 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:41.181912 27383 external_mini_cluster.cc:1428] Setting key 446bfb0854806086936fea508b187640
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:41.191603 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.179896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:38057 (local address 127.26.189.254:44523)
0807 22:42:41.180036 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.180040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.180846 (+ 806us) server_negotiation.cc:407] Connection header received
0807 22:42:41.182204 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.182208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.182265 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.182366 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:41.183561 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.184076 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.185173 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.185404 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.185491 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.188241 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:41.188244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:41.188246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:41.188269 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:41.189629 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.190089 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.190093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.190095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.190169 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.190516 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.190520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.190522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.190699 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:41.190787 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.191313 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.191434 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:42:41.192373 28757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.180234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44523 (local address 127.26.189.194:38057)
0807 22:42:41.180695 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.180737 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.182000 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.182546 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.182553 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.182882 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:41.183423 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.183439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.184196 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.184199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.185052 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.185062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.185541 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.185546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.185699 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.186282 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:41.186322 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:41.188054 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:41.189736 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:41.189741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:41.189756 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:41.189994 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:41.190251 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:41.190255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:41.190257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:41.190427 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:41.190799 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:41.190805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:41.191079 (+ 274us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.191252 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.191549 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":130,"threads_started":1}
I20260807 22:42:41.193454 28754 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44523
I20260807 22:42:41.193673 28754 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:41.194166 28754 heartbeater.cc:511] Master 127.26.189.254:44523 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
I20260807 22:42:41.195251 28263 ts_manager.cc:194] Registered new tserver with Master: 6234d7309a1b461194ec7b1866b87aa8 (127.26.189.194:40057)
I20260807 22:42:41.195796 28263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:38057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:41.209549 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44523
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:41.317559 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:41.317850 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:41.317912 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:41.321915 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:41.321969 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:41.322048 28761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:41.327191 28761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44523
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:41.328156 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:41.329152 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:41.338387 28768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:41.338446 28772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:41.338629 28761 server_base.cc:1034] running on GCE node
W20260807 22:42:41.338478 28769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:41.339064 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:41.339644 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:41.340801 28761 hybrid_clock.cc:648] HybridClock initialized: now 1786142561340784 us; error 27 us; skew 500 ppm
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:41.343365 28761 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:41.344190 28761 webserver.cc:460] Webserver started at http://127.26.189.195:38327/ using document root <none> and password file <none>
I20260807 22:42:41.344717 28761 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:41.344784 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:41.344975 28761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:41.347054 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "991a0d69499d4f7386f864adffefde78"
format_stamp: "Formatted at 2026-08-07 22:42:41 on dist-test-slave-4tdj"
server_key: "78b1064f271738e8660fff76d3415154"
server_key_iv: "14dcad0042c22d617eb2ece6c068b737"
server_key_version: "encryptionkey@0"
I20260807 22:42:41.347586 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "991a0d69499d4f7386f864adffefde78"
format_stamp: "Formatted at 2026-08-07 22:42:41 on dist-test-slave-4tdj"
server_key: "78b1064f271738e8660fff76d3415154"
server_key_iv: "14dcad0042c22d617eb2ece6c068b737"
server_key_version: "encryptionkey@0"
I20260807 22:42:41.351015 28761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:42:41.353214 28783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:41.354321 28761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:42:41.354431 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "991a0d69499d4f7386f864adffefde78"
format_stamp: "Formatted at 2026-08-07 22:42:41 on dist-test-slave-4tdj"
server_key: "78b1064f271738e8660fff76d3415154"
server_key_iv: "14dcad0042c22d617eb2ece6c068b737"
server_key_version: "encryptionkey@0"
I20260807 22:42:41.354516 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:41.372792 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:41.375473 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:41.375638 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:41.376196 28761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:41.376958 28761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:41.377004 28761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:41.377055 28761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:41.377080 28761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:41.385685 28761 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:41855
I20260807 22:42:41.385720 28946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:41855 every 8 connection(s)
I20260807 22:42:41.386556 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:41.395510 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28761
I20260807 22:42:41.395643 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:41.395915 27383 external_mini_cluster.cc:1428] Setting key 529b2c650d3d12c24c25d55cf96b7b7e
I20260807 22:42:41.398901 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.388223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:35827 (local address 127.26.189.254:44523)
0807 22:42:41.388344 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.388349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.389132 (+ 783us) server_negotiation.cc:407] Connection header received
0807 22:42:41.390324 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.390327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.390374 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.390444 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:41.391481 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.391982 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.392995 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.393219 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.393292 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.395581 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:41.395586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:41.395589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:41.395617 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:41.397018 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.397471 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.397475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.397477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.397557 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.397859 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.397862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.397864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.398046 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:41.398127 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.398633 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.398747 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:42:41.399626 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.388538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44523 (local address 127.26.189.195:35827)
0807 22:42:41.388982 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.389022 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.390125 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.390585 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.390609 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.390932 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:41.391369 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.391381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.392066 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.392068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.392903 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.392911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.393326 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.393328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.393459 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.393937 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:41.393952 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:41.395414 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:41.397126 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:41.397131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:41.397144 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:41.397379 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:41.397639 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:41.397643 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:41.397645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:41.397772 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:41.398150 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:41.398156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:41.398408 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.398566 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.398864 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
I20260807 22:42:41.400547 28947 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44523
I20260807 22:42:41.400741 28947 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:41.401160 28947 heartbeater.cc:511] Master 127.26.189.254:44523 requested a full tablet report, sending...
I20260807 22:42:41.402025 28263 ts_manager.cc:194] Registered new tserver with Master: 991a0d69499d4f7386f864adffefde78 (127.26.189.195:41855)
I20260807 22:42:41.402614 28263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:35827
I20260807 22:42:41.409338 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:42:41.418699 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.412590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.26.189.254:44523)
0807 22:42:41.412699 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.412703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.412853 (+ 150us) server_negotiation.cc:407] Connection header received
0807 22:42:41.412940 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.412942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.412977 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.413038 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:41.413648 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.414131 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.414973 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.415186 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.415264 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.416399 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:41.416402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:41.416403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:41.416419 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:41.417432 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.417764 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.417766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.417767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.417831 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.418007 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.418009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.418010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.418191 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:41.418286 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.418490 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.418579 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:42:41.422724 28263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:41.425494 28263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:41.439621 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.434131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.26.189.193:45589)
0807 22:42:41.434721 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.434741 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.434765 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:42:41.434846 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.434852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.435054 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.435215 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:41.435700 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.436782 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.438350 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.438948 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.439069 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.439201 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.439392 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.439486 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":171,"threads_started":1}
I20260807 22:42:41.439612 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.433239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:45589 (local address 127.0.0.1:37574)
0807 22:42:41.433556 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.433601 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.433728 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.435401 (+ 1673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.435405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.435425 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:41.435568 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.435577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.436937 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.436940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.438108 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.438115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.439084 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.439088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.439209 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.439250 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.439352 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.439468 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":30,"thread_start_us":68,"threads_started":1}
I20260807 22:42:41.439944 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.433424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:40057 (local address 127.0.0.1:55982)
0807 22:42:41.434075 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.434091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.434476 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.435154 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.435159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.435182 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:41.435319 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.435327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.436650 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.436653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.438087 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.438100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.439349 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.439353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.439588 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.439621 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.439701 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.439786 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":119,"thread_start_us":138,"threads_started":1}
I20260807 22:42:41.440112 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.434072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.26.189.195:41855)
0807 22:42:41.434497 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.434509 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.434909 (+ 400us) server_negotiation.cc:407] Connection header received
0807 22:42:41.434988 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.434993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.435159 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.435291 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:41.435725 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.436474 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.438109 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.439007 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.439151 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.439278 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.439869 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.439966 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":103,"threads_started":1}
I20260807 22:42:41.440471 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.434026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.26.189.194:40057)
0807 22:42:41.434680 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.434691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.434714 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:42:41.434793 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.434799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.435000 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.435156 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:41.435425 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.436503 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.438226 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.439036 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.439151 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.439278 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.440211 (+ 933us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.440318 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":122,"threads_started":1}
I20260807 22:42:41.440721 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.433672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:41855 (local address 127.0.0.1:50324)
0807 22:42:41.434700 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.434715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.434811 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.435468 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.435471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.435486 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:41.435597 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.435603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.436608 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.436612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.437986 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.437999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.439349 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.439353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.439583 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.439630 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.439701 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.439786 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":168,"thread_start_us":100,"threads_started":1}
I20260807 22:42:41.442755 28448 tablet_service.cc:1467] Processing CreateTablet for tablet 84fd53ea984a4a6d8a797f7fa5d529fa (DEFAULT_TABLE table=test-table [id=037b00bd3ebe4515866bb96741df9bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:41.443676 28448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84fd53ea984a4a6d8a797f7fa5d529fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:41.443372 28852 tablet_service.cc:1467] Processing CreateTablet for tablet 84fd53ea984a4a6d8a797f7fa5d529fa (DEFAULT_TABLE table=test-table [id=037b00bd3ebe4515866bb96741df9bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:41.444435 28852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84fd53ea984a4a6d8a797f7fa5d529fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:41.444085 28647 tablet_service.cc:1467] Processing CreateTablet for tablet 84fd53ea984a4a6d8a797f7fa5d529fa (DEFAULT_TABLE table=test-table [id=037b00bd3ebe4515866bb96741df9bac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:41.445067 28647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84fd53ea984a4a6d8a797f7fa5d529fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:41.449858 28966 tablet_bootstrap.cc:492] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35: Bootstrap starting.
I20260807 22:42:41.451611 28967 tablet_bootstrap.cc:492] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: Bootstrap starting.
I20260807 22:42:41.452008 28966 tablet_bootstrap.cc:654] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:41.452486 28970 tablet_bootstrap.cc:492] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8: Bootstrap starting.
I20260807 22:42:41.453275 28966 log.cc:824] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:41.453771 28967 tablet_bootstrap.cc:654] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:41.454687 28970 tablet_bootstrap.cc:654] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:41.454952 28967 log.cc:824] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:41.455513 28966 tablet_bootstrap.cc:492] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35: No bootstrap required, opened a new log
I20260807 22:42:41.455597 28970 log.cc:824] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:41.455690 28966 ts_tablet_manager.cc:1397] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:42:41.457006 28967 tablet_bootstrap.cc:492] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: No bootstrap required, opened a new log
I20260807 22:42:41.457209 28967 ts_tablet_manager.cc:1397] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.005s
I20260807 22:42:41.457391 28970 tablet_bootstrap.cc:492] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8: No bootstrap required, opened a new log
I20260807 22:42:41.457566 28970 ts_tablet_manager.cc:1397] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:42:41.459488 28966 raft_consensus.cc:348] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.459748 28966 raft_consensus.cc:374] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:41.459810 28966 raft_consensus.cc:729] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415590a7b83642488ebfe50f419abc35, State: Initialized, Role: FOLLOWER
I20260807 22:42:41.460243 28966 consensus_queue.cc:260] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.460915 28552 heartbeater.cc:503] Master 127.26.189.254:44523 was elected leader, sending a full tablet report...
I20260807 22:42:41.460853 28967 raft_consensus.cc:348] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.461100 28967 raft_consensus.cc:374] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:41.461158 28967 raft_consensus.cc:729] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991a0d69499d4f7386f864adffefde78, State: Initialized, Role: FOLLOWER
I20260807 22:42:41.461236 28966 ts_tablet_manager.cc:1428] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:42:41.461570 28967 consensus_queue.cc:260] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.462427 28967 ts_tablet_manager.cc:1428] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:42:41.462435 28947 heartbeater.cc:503] Master 127.26.189.254:44523 was elected leader, sending a full tablet report...
I20260807 22:42:41.460729 28970 raft_consensus.cc:348] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.463847 28970 raft_consensus.cc:374] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:41.463914 28970 raft_consensus.cc:729] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6234d7309a1b461194ec7b1866b87aa8, State: Initialized, Role: FOLLOWER
I20260807 22:42:41.464310 28970 consensus_queue.cc:260] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.464942 28754 heartbeater.cc:503] Master 127.26.189.254:44523 was elected leader, sending a full tablet report...
I20260807 22:42:41.466208 28970 ts_tablet_manager.cc:1428] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260807 22:42:41.542672 28976 raft_consensus.cc:482] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:41.542876 28976 raft_consensus.cc:504] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.544008 28976 leader_election.cc:288] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6234d7309a1b461194ec7b1866b87aa8 (127.26.189.194:40057), 991a0d69499d4f7386f864adffefde78 (127.26.189.195:41855)
I20260807 22:42:41.547802 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.544339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:54903 (local address 127.26.189.194:40057)
0807 22:42:41.544548 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.544553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.544607 (+ 54us) server_negotiation.cc:407] Connection header received
0807 22:42:41.544748 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.544751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.544804 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.544874 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:41.545189 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.545719 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.546860 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.547263 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.547369 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.547423 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.547603 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.547675 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260807 22:42:41.547813 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.544383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:43269 (local address 127.26.189.195:41855)
0807 22:42:41.544533 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.544538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.544766 (+ 228us) server_negotiation.cc:407] Connection header received
0807 22:42:41.544880 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.544883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.544932 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.545003 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:41.545247 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.545730 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.546864 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.547272 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.547371 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.547423 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.547645 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.547714 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:42:41.547825 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.544272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:41855 (local address 127.26.189.193:43269)
0807 22:42:41.544676 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.544688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.544789 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.545020 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.545022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.545036 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:41.545142 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.545149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.545830 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.545833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.546752 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.546762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.547377 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.547381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.547524 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.547542 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.547618 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.547682 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":71,"thread_start_us":121,"threads_started":1}
I20260807 22:42:41.547832 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.544197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:40057 (local address 127.26.189.193:54903)
0807 22:42:41.544529 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:41.544543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:41.544649 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:41.544905 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:41.544908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:41.544925 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:41.545081 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.545088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.545823 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.545826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.546747 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:41.546757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.547371 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.547375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:41.547485 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.547507 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:42:41.547565 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:42:41.547633 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":21,"thread_start_us":100,"threads_started":1}
I20260807 22:42:41.548462 28879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84fd53ea984a4a6d8a797f7fa5d529fa" candidate_uuid: "415590a7b83642488ebfe50f419abc35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991a0d69499d4f7386f864adffefde78" is_pre_election: true
I20260807 22:42:41.548548 28681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84fd53ea984a4a6d8a797f7fa5d529fa" candidate_uuid: "415590a7b83642488ebfe50f419abc35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6234d7309a1b461194ec7b1866b87aa8" is_pre_election: true
I20260807 22:42:41.548758 28879 raft_consensus.cc:2454] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 415590a7b83642488ebfe50f419abc35 in term 0.
I20260807 22:42:41.548817 28681 raft_consensus.cc:2454] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 415590a7b83642488ebfe50f419abc35 in term 0.
I20260807 22:42:41.549185 28382 leader_election.cc:302] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 415590a7b83642488ebfe50f419abc35, 991a0d69499d4f7386f864adffefde78; no voters:
I20260807 22:42:41.549396 28976 raft_consensus.cc:2781] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:41.549485 28976 raft_consensus.cc:482] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:41.549546 28976 raft_consensus.cc:3037] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:41.550638 28976 raft_consensus.cc:504] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.550983 28976 leader_election.cc:288] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [CANDIDATE]: Term 1 election: Requested vote from peers 6234d7309a1b461194ec7b1866b87aa8 (127.26.189.194:40057), 991a0d69499d4f7386f864adffefde78 (127.26.189.195:41855)
I20260807 22:42:41.551303 28681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84fd53ea984a4a6d8a797f7fa5d529fa" candidate_uuid: "415590a7b83642488ebfe50f419abc35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6234d7309a1b461194ec7b1866b87aa8"
I20260807 22:42:41.551298 28879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84fd53ea984a4a6d8a797f7fa5d529fa" candidate_uuid: "415590a7b83642488ebfe50f419abc35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991a0d69499d4f7386f864adffefde78"
I20260807 22:42:41.551410 28879 raft_consensus.cc:3037] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:41.551412 28681 raft_consensus.cc:3037] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:41.552387 28879 raft_consensus.cc:2454] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 415590a7b83642488ebfe50f419abc35 in term 1.
I20260807 22:42:41.552410 28681 raft_consensus.cc:2454] T 84fd53ea984a4a6d8a797f7fa5d529fa P 6234d7309a1b461194ec7b1866b87aa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 415590a7b83642488ebfe50f419abc35 in term 1.
I20260807 22:42:41.552655 28382 leader_election.cc:302] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 415590a7b83642488ebfe50f419abc35, 991a0d69499d4f7386f864adffefde78; no voters:
I20260807 22:42:41.552811 28976 raft_consensus.cc:2781] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:41.553014 28976 raft_consensus.cc:686] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [term 1 LEADER]: Becoming Leader. State: Replica: 415590a7b83642488ebfe50f419abc35, State: Running, Role: LEADER
I20260807 22:42:41.553328 28976 consensus_queue.cc:237] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } }
I20260807 22:42:41.556213 28263 catalog_manager.cc:5495] T 84fd53ea984a4a6d8a797f7fa5d529fa P 415590a7b83642488ebfe50f419abc35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 415590a7b83642488ebfe50f419abc35 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "415590a7b83642488ebfe50f419abc35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "415590a7b83642488ebfe50f419abc35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6234d7309a1b461194ec7b1866b87aa8" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991a0d69499d4f7386f864adffefde78" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41855 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:41.591315 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.583472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.26.189.193:45589)
0807 22:42:41.583616 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.583621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.583750 (+ 129us) server_negotiation.cc:407] Connection header received
0807 22:42:41.583851 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.583853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.583895 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.583958 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:41.584593 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.585078 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.585836 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.586031 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.586098 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.588318 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:41.588332 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:41.588346 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:41.588370 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:41.589884 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.590220 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.590225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.590226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.590354 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.590549 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.590551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.590552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.590782 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:41.590924 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.591048 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.591154 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:41.616729 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.609349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37586 (local address 127.26.189.193:45589)
0807 22:42:41.609485 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.609491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.609591 (+ 100us) server_negotiation.cc:407] Connection header received
0807 22:42:41.609710 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.609713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.609755 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.609820 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:41.610520 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.611061 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.611804 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.612021 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.612089 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.614164 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:41.614168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:41.614170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:41.614194 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:41.615447 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.615790 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.615792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.615793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.615859 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.616036 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.616038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.616039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.616191 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:41.616300 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.616481 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.616571 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260807 22:42:41.637468 28948 tablet.cc:2369] T 84fd53ea984a4a6d8a797f7fa5d529fa P 991a0d69499d4f7386f864adffefde78: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:41.640724 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:41.633673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.26.189.193:45589)
0807 22:42:41.633803 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:41.633807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:41.633951 (+ 144us) server_negotiation.cc:407] Connection header received
0807 22:42:41.634046 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:41.634049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:41.634089 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:41.634159 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:41.634793 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.635283 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.636007 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:41.636209 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:41.636274 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:41.638507 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:41.638510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:41.638512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:41.638534 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:41.639568 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.639900 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.639902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.639903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.639967 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:41.640149 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:41.640151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:41.640152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:41.640288 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:41.640395 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:41.640525 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 22:42:41.640611 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 22:42:41.641927 28419 tablet_service.cc:2908] Rejecting scan request for tablet 84fd53ea984a4a6d8a797f7fa5d529fa: Uninitialized: clean time has not yet been initialized
I20260807 22:42:41.644022 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28357
I20260807 22:42:41.657282 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28560
I20260807 22:42:41.669350 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28761
I20260807 22:42:41.677712 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28208
2026-08-07T22:42:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2433 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28999](info): set up 2 sockets
Aug 07 22:42:41 dist-test-slave-4tdj krb5kdc[28999](info): commencing operation
krb5kdc: starting...
W20260807 22:42:42.828987 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:42 dist-test-slave-4tdj krb5kdc[28999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:42.946959 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:34985
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:34985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:43.057129 29018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:43.057394 29018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:43.057471 29018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:43.061561 29018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:43.061630 29018 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:42:43.061658 29018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:43.061687 29018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:43.061724 29018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:43.061755 29018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:43.067427 29018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:34985
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:34985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:43.068442 29018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:43.069415 29018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:43.077522 29025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.077732 29018 server_base.cc:1034] running on GCE node
W20260807 22:42:43.077564 29028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.077530 29024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.078418 29018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:43.079456 29018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:43.080632 29018 hybrid_clock.cc:648] HybridClock initialized: now 1786142563080617 us; error 28 us; skew 500 ppm
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:43.086009 29018 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:43.086886 29018 webserver.cc:460] Webserver started at http://127.26.189.254:36927/ using document root <none> and password file <none>
I20260807 22:42:43.087399 29018 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:43.087463 29018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:43.087697 29018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:43.089664 29018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "980200fbe79044c88043caed8d8890c7"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "648627a6551602ea164583e3c0e98c88"
server_key_iv: "9a703fc493063549c828e790c3afc407"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.090181 29018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "980200fbe79044c88043caed8d8890c7"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "648627a6551602ea164583e3c0e98c88"
server_key_iv: "9a703fc493063549c828e790c3afc407"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.093793 29018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:42:43.096071 29038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.097205 29018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:43.097314 29018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "980200fbe79044c88043caed8d8890c7"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "648627a6551602ea164583e3c0e98c88"
server_key_iv: "9a703fc493063549c828e790c3afc407"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.097402 29018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:43.110332 29018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:43.113220 29018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:43.113375 29018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:43.120949 29018 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:34985
I20260807 22:42:43.120993 29140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:34985 every 8 connection(s)
I20260807 22:42:43.121821 29018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:43.122470 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29018
I20260807 22:42:43.122573 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:43.122807 27383 external_mini_cluster.cc:1428] Setting key 4eac0d8c7f3c28c03c6fa9c9eac3a6a2
I20260807 22:42:43.124810 29143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:43.131219 29143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: Bootstrap starting.
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:43.133832 29143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:43.135135 29143 log.cc:824] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:43.137015 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.124112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.26.189.254:34985)
0807 22:42:43.124523 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.124543 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.124584 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:42:43.125415 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.125437 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.125839 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.126184 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.126859 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.128337 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.129182 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.129844 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.129994 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.132294 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.132300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.132317 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.132348 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.134224 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.134703 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.134709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.134714 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.134852 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.135079 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.135082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.135083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.135574 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.135768 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.136050 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.136325 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":119,"threads_started":1}
I20260807 22:42:43.137050 29143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: No bootstrap required, opened a new log
I20260807 22:42:43.140412 29143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980200fbe79044c88043caed8d8890c7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 34985 } }
I20260807 22:42:43.140671 29143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:43.140733 29143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 980200fbe79044c88043caed8d8890c7, State: Initialized, Role: FOLLOWER
I20260807 22:42:43.141151 29143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980200fbe79044c88043caed8d8890c7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 34985 } }
I20260807 22:42:43.141281 29143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:43.141343 29143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:43.141435 29143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:43.142282 29143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980200fbe79044c88043caed8d8890c7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 34985 } }
I20260807 22:42:43.142616 29143 leader_election.cc:302] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 980200fbe79044c88043caed8d8890c7; no voters:
I20260807 22:42:43.142925 29143 leader_election.cc:288] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:43.143020 29151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:43.143255 29151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [term 1 LEADER]: Becoming Leader. State: Replica: 980200fbe79044c88043caed8d8890c7, State: Running, Role: LEADER
I20260807 22:42:43.143584 29151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980200fbe79044c88043caed8d8890c7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 34985 } }
I20260807 22:42:43.144021 29143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:43.145138 29152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "980200fbe79044c88043caed8d8890c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980200fbe79044c88043caed8d8890c7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 34985 } } }
I20260807 22:42:43.145192 29154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980200fbe79044c88043caed8d8890c7. Latest consensus state: current_term: 1 leader_uuid: "980200fbe79044c88043caed8d8890c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980200fbe79044c88043caed8d8890c7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 34985 } } }
I20260807 22:42:43.145278 29152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:43.145299 29154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:43.145633 29163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:43.148557 29163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:43.153928 29163 catalog_manager.cc:1347] Generated new cluster ID: 975d4c5629be45ec8464e4856ad344a3
I20260807 22:42:43.153978 29163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:43.166194 29163 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:43.167433 29163 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:43.172859 29163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: Generated new TSK 0
I20260807 22:42:43.173566 29163 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:43.199379 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:34985
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:43.307861 29181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:43.308108 29181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:43.308169 29181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:43.312142 29181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:43.312196 29181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:43.312249 29181 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:43.312305 29181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:43.317358 29181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:34985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:43.318367 29181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:43.319342 29181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:43.328341 29192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.328395 29181 server_base.cc:1034] running on GCE node
W20260807 22:42:43.328361 29189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.328351 29190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.328986 29181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:43.329571 29181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:43.330724 29181 hybrid_clock.cc:648] HybridClock initialized: now 1786142563330711 us; error 28 us; skew 500 ppm
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:43.333249 29181 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:43.334017 29181 webserver.cc:460] Webserver started at http://127.26.189.193:41073/ using document root <none> and password file <none>
I20260807 22:42:43.334554 29181 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:43.334625 29181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:43.334833 29181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:43.336720 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "8f9d0f41d4a748658af23acdc5798658"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "ae976aeddecc9bcec2e5c4d162f7b252"
server_key_iv: "d59c02ccb17e5b88b62a79ff3fcafdc8"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.337232 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "8f9d0f41d4a748658af23acdc5798658"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "ae976aeddecc9bcec2e5c4d162f7b252"
server_key_iv: "d59c02ccb17e5b88b62a79ff3fcafdc8"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.340677 29181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:42:43.342902 29204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.344043 29181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:43.344148 29181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "8f9d0f41d4a748658af23acdc5798658"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "ae976aeddecc9bcec2e5c4d162f7b252"
server_key_iv: "d59c02ccb17e5b88b62a79ff3fcafdc8"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.344261 29181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:43.358412 29181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:43.361342 29181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:43.361492 29181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:43.362859 29181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:43.362911 29181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.362967 29181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:43.362994 29181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:43.372933 29181 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42191
I20260807 22:42:43.373343 29367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42191 every 8 connection(s)
I20260807 22:42:43.373872 29181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:43.375372 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29181
I20260807 22:42:43.375489 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:43.375795 27383 external_mini_cluster.cc:1428] Setting key 84bd40c7f4e6b1e4e8cfeefb48dd9878
I20260807 22:42:43.376080 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.363636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:47931 (local address 127.26.189.254:34985)
0807 22:42:43.363791 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.363797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.364569 (+ 772us) server_negotiation.cc:407] Connection header received
0807 22:42:43.365774 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.365778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.365840 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.365940 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.367213 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.368028 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.369248 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.369612 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.369717 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.372486 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.372491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.372494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.372523 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.373966 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.374512 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.374515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.374517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.374627 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.374972 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.374974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.374975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.375176 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.375250 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.375833 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.375959 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:42:43.376814 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.363961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.26.189.193:47931)
0807 22:42:43.364403 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.364454 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.365539 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.366145 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.366154 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.366583 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.367064 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.367078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.368146 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.368150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.369127 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.369137 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.369769 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.369774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.369991 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.370548 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.370569 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.372319 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.374068 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.374074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.374088 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.374398 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.374707 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.374709 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.374711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.374877 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.375266 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.375272 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.375561 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.375780 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.376088 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
I20260807 22:42:43.378144 29369 heartbeater.cc:347] Connected to a master server at 127.26.189.254:34985
I20260807 22:42:43.378285 29369 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:43.378845 29369 heartbeater.cc:511] Master 127.26.189.254:34985 requested a full tablet report, sending...
I20260807 22:42:43.380741 29063 ts_manager.cc:194] Registered new tserver with Master: 8f9d0f41d4a748658af23acdc5798658 (127.26.189.193:42191)
I20260807 22:42:43.381968 29063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:47931
I20260807 22:42:43.387398 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.380249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:44151 (local address 127.26.189.254:34985)
0807 22:42:43.380396 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.380401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.380625 (+ 224us) server_negotiation.cc:407] Connection header received
0807 22:42:43.380779 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.380782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.380823 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.380866 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.381475 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.382008 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.383053 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.383382 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.383491 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.384672 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.384674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.384677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.384711 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.385951 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.386411 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.386414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.386415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.386482 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.386742 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.386744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.386745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.386887 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.386954 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.387207 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.387293 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:42:43.387486 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.380265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.26.189.193:44151)
0807 22:42:43.380549 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.380563 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.380688 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.381006 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.381009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.381216 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.381338 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.381345 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.382156 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.382159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.382932 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.382944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.383518 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.383522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.383623 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.384128 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.384143 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.384541 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.386055 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.386058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.386060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.386324 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.386554 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.386557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.386559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.386664 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.386968 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.386972 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.387065 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.387172 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.387302 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:43.404785 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:34985
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:43.513803 29382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:43.514052 29382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:43.514099 29382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:43.518041 29382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:43.518096 29382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:43.518150 29382 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:43.518205 29382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:43.523340 29382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:34985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:43.524307 29382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:43.525303 29382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:43.534462 29395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.534462 29393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.534497 29382 server_base.cc:1034] running on GCE node
W20260807 22:42:43.534534 29392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.535102 29382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:43.535727 29382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:43.536911 29382 hybrid_clock.cc:648] HybridClock initialized: now 1786142563536894 us; error 28 us; skew 500 ppm
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:43.539501 29382 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:43.540319 29382 webserver.cc:460] Webserver started at http://127.26.189.194:34079/ using document root <none> and password file <none>
I20260807 22:42:43.540838 29382 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:43.540903 29382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:43.541100 29382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:43.543102 29382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "a9090f8d1fc34b4abdc143e1fc3d3196"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "0b46f13baff6212df25848b6eb121535"
server_key_iv: "f0d1e5c91548d16fe1f8ba6006512d6d"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.543671 29382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "a9090f8d1fc34b4abdc143e1fc3d3196"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "0b46f13baff6212df25848b6eb121535"
server_key_iv: "f0d1e5c91548d16fe1f8ba6006512d6d"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.547271 29382 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:42:43.549525 29403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.550688 29382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:42:43.550791 29382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "a9090f8d1fc34b4abdc143e1fc3d3196"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "0b46f13baff6212df25848b6eb121535"
server_key_iv: "f0d1e5c91548d16fe1f8ba6006512d6d"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.550876 29382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:43.573083 29382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:43.575664 29382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:43.575830 29382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:43.577193 29382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:43.577241 29382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.577293 29382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:43.577320 29382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:43.587664 29382 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:45683
I20260807 22:42:43.587726 29586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:45683 every 8 connection(s)
I20260807 22:42:43.588655 29382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:43.590549 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29382
I20260807 22:42:43.590672 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:43.590883 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.577958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53063 (local address 127.26.189.254:34985)
0807 22:42:43.578138 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.578143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.578971 (+ 828us) server_negotiation.cc:407] Connection header received
0807 22:42:43.580238 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.580241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.580304 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.580363 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.582057 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.582767 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.584434 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.584749 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.584841 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.587588 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.587592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.587594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.587623 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.588943 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.589522 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.589525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.589527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.589612 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.589888 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.589890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.589891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.590054 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.590141 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.590632 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.590763 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:42:43.590976 27383 external_mini_cluster.cc:1428] Setting key 216cdb1185dc0b07d872629cc1383f1f
I20260807 22:42:43.591777 29415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.578328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.26.189.194:53063)
0807 22:42:43.578798 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.578855 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.579949 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.580788 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.580798 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.581252 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.581906 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.581926 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.582901 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.582905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.584309 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.584322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.584900 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.584905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.585099 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.585695 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.585718 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.587406 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.589060 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.589066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.589079 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.589405 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.589691 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.589693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.589695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.589820 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.590125 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.590129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.590391 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.590612 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.590921 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":134,"threads_started":1}
I20260807 22:42:43.593041 29587 heartbeater.cc:347] Connected to a master server at 127.26.189.254:34985
I20260807 22:42:43.593194 29587 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:43.593734 29587 heartbeater.cc:511] Master 127.26.189.254:34985 requested a full tablet report, sending...
I20260807 22:42:43.594890 29064 ts_manager.cc:194] Registered new tserver with Master: a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683)
I20260807 22:42:43.595638 29064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:53063
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:42:43.602244 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.595040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.26.189.194:33379)
0807 22:42:43.595321 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.595334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.595415 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.595732 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.595735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.595902 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.595990 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.595995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.596924 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.596927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.597696 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.597708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.598243 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.598245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.598340 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.598799 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.598809 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.599110 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.600837 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.600840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.600842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.601023 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.601306 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.601308 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.601309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.601382 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.601844 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.601846 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.601921 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.601999 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.602109 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":112,"threads_started":1}
I20260807 22:42:43.602412 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.595380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:33379 (local address 127.26.189.254:34985)
0807 22:42:43.595483 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.595487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.595498 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:42:43.595527 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.595530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.595565 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.595626 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.596098 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.596777 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.597825 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.598146 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.598249 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.599244 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.599249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.599251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.599278 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.600738 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.601128 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.601131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.601133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.601217 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.601458 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.601462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.601463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.601755 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.601857 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.602067 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.602241 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:43.620388 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:34985
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:43.729606 29606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:43.729839 29606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:43.729882 29606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:43.733804 29606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:43.733857 29606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:43.733897 29606 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:43.733949 29606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:43.739117 29606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:34985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:43.740057 29606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:43.741021 29606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:43.750623 29612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.750708 29614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:43.750712 29616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:43.750830 29606 server_base.cc:1034] running on GCE node
I20260807 22:42:43.751273 29606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:43.751884 29606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:43.753047 29606 hybrid_clock.cc:648] HybridClock initialized: now 1786142563753029 us; error 34 us; skew 500 ppm
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:43.755728 29606 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:43.756582 29606 webserver.cc:460] Webserver started at http://127.26.189.195:41181/ using document root <none> and password file <none>
I20260807 22:42:43.757067 29606 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:43.757131 29606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:43.757289 29606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:43.759303 29606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "2ce73b3505ce42d1b6e07b9763d41acb"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "caa202e9a1258151e3a68d38a40a0c71"
server_key_iv: "3f7e7a35bcee2d78ff9fb72a4d8f72eb"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.759790 29606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "2ce73b3505ce42d1b6e07b9763d41acb"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "caa202e9a1258151e3a68d38a40a0c71"
server_key_iv: "3f7e7a35bcee2d78ff9fb72a4d8f72eb"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.763298 29606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:42:43.765511 29629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.766692 29606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:43.766801 29606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "2ce73b3505ce42d1b6e07b9763d41acb"
format_stamp: "Formatted at 2026-08-07 22:42:43 on dist-test-slave-4tdj"
server_key: "caa202e9a1258151e3a68d38a40a0c71"
server_key_iv: "3f7e7a35bcee2d78ff9fb72a4d8f72eb"
server_key_version: "encryptionkey@0"
I20260807 22:42:43.766893 29606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:43.780499 29606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:43.783432 29606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:43.783610 29606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:43.785066 29606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:43.785123 29606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:43.785164 29606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:43.785192 29606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:43.794983 29606 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:43879
I20260807 22:42:43.795042 29805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:43879 every 8 connection(s)
I20260807 22:42:43.795930 29606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:43.796443 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29606
I20260807 22:42:43.796563 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:43.796830 27383 external_mini_cluster.cc:1428] Setting key e08828c38b0fab7bc98ca7128e20265b
I20260807 22:42:43.799176 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.785843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:35035 (local address 127.26.189.254:34985)
0807 22:42:43.785991 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.785996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.786949 (+ 953us) server_negotiation.cc:407] Connection header received
0807 22:42:43.788008 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.788012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.788072 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.788149 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.789537 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.790388 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.792047 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.792272 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.792355 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.795433 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.795437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.795440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.795469 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.797044 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.797635 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.797639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.797640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.797742 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.798119 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.798122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.798124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.798368 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.798464 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.798929 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.799057 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:42:43.799923 29641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.786281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.26.189.195:35035)
0807 22:42:43.786785 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.786834 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.787742 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.788324 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.788334 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.788752 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.789390 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.789411 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.790530 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.790534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.791928 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.791941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.792429 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.792433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.792650 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.793306 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.793327 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.795273 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.797165 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.797171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.797185 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.797506 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.797841 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.797844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.797846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.798027 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.798457 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.798463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.798691 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.798865 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.799170 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260807 22:42:43.800973 29806 heartbeater.cc:347] Connected to a master server at 127.26.189.254:34985
I20260807 22:42:43.801108 29806 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:43.801659 29806 heartbeater.cc:511] Master 127.26.189.254:34985 requested a full tablet report, sending...
I20260807 22:42:43.802731 29063 ts_manager.cc:194] Registered new tserver with Master: 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:43.803447 29063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:35035
I20260807 22:42:43.809212 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.803265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:50947 (local address 127.26.189.254:34985)
0807 22:42:43.803367 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.803371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.803538 (+ 167us) server_negotiation.cc:407] Connection header received
0807 22:42:43.803635 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.803638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.803683 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.803749 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.804217 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.804940 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.805685 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.805888 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.805962 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.806930 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.806933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.806935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.806952 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.808006 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.808341 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.808343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.808344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.808406 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.808586 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.808588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.808589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.808732 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.808799 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.809000 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.809080 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:42:43.809237 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.803211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.26.189.195:50947)
0807 22:42:43.803466 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.803478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.803563 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.803784 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.803787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.803991 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:43.804105 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.804111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.805041 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.805043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.805590 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.805598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.805987 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.805990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.806058 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.806527 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:43.806538 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:43.806833 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:43.808098 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.808100 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.808102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.808273 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.808465 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:43.808467 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:43.808468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:43.808530 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:43.808809 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:43.808811 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:43.808884 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.808959 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.809054 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260807 22:42:43.810559 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:43 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:43.834534 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.827481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.26.189.254:34985)
0807 22:42:43.827610 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.827614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.827625 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:42:43.827663 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.827666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.827704 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.827763 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:43.828410 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.828923 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.829662 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.829859 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.829927 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.832029 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:43.832033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:43.832035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:43.832057 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:43.833287 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.833604 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.833606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.833608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.833674 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:43.833850 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:43.833852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:43.833853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:43.834007 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:43.834104 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.834252 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.834375 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:42:43.837352 29063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:43.840391 29063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:43.854334 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.848578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42191 (local address 127.0.0.1:51392)
0807 22:42:43.848967 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.849018 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.849175 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.849906 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.849912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.849936 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:43.850066 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.850074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.851388 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.851391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.852728 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.852739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.853799 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.853803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.853909 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.853948 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.854046 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.854164 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
I20260807 22:42:43.854346 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.848696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.26.189.193:42191)
0807 22:42:43.849009 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.849036 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.849090 (+ 54us) server_negotiation.cc:407] Connection header received
0807 22:42:43.849337 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.849343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.849541 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.849715 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:43.850205 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.851243 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.852855 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.853698 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.853866 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.854017 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.854068 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.854168 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":121,"threads_started":1}
I20260807 22:42:43.854691 29835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.849350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:45683 (local address 127.0.0.1:41742)
0807 22:42:43.849585 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.849606 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.849688 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.850871 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.850873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.850890 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:43.850994 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.851000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.852267 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.852271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.853228 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.853237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.854327 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.854332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.854424 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.854450 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.854498 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.854558 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":40,"thread_start_us":102,"threads_started":1}
I20260807 22:42:43.855480 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.850123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.26.189.194:45683)
0807 22:42:43.850442 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.850453 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.850475 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:42:43.850552 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.850557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.850750 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.850883 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:43.851083 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.852137 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.853360 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.854165 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.855070 (+ 905us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.855200 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.855253 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.855341 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":106,"threads_started":1}
I20260807 22:42:43.857277 29275 tablet_service.cc:1467] Processing CreateTablet for tablet 8d59750943d84e0f98faea937fbd6524 (DEFAULT_TABLE table=test-table [id=cd8a295c5f6745108ba97cbbb919ab54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:43.857676 29480 tablet_service.cc:1467] Processing CreateTablet for tablet 8d59750943d84e0f98faea937fbd6524 (DEFAULT_TABLE table=test-table [id=cd8a295c5f6745108ba97cbbb919ab54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:43.858042 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.849678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:43879 (local address 127.0.0.1:56140)
0807 22:42:43.850511 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.850523 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.850623 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.851174 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.851177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.851194 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:43.851287 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.851293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.852290 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.852293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.853531 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.853543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.855100 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.855104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.855188 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.855209 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.857822 (+ 2613us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.857903 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":20,"thread_start_us":92,"threads_started":1}
I20260807 22:42:43.858361 29275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d59750943d84e0f98faea937fbd6524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:43.858563 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.849914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56140 (local address 127.26.189.195:43879)
0807 22:42:43.850259 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.850273 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.850621 (+ 348us) server_negotiation.cc:407] Connection header received
0807 22:42:43.850722 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.850727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.850897 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.851030 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:43.851408 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.852166 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.853656 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.854760 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.854904 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.855036 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.858330 (+ 3294us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.858431 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":118,"threads_started":1}
I20260807 22:42:43.858634 29480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d59750943d84e0f98faea937fbd6524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:43.864240 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 8d59750943d84e0f98faea937fbd6524 (DEFAULT_TABLE table=test-table [id=cd8a295c5f6745108ba97cbbb919ab54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:43.865108 29844 tablet_bootstrap.cc:492] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Bootstrap starting.
I20260807 22:42:43.865295 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d59750943d84e0f98faea937fbd6524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:43.866359 29845 tablet_bootstrap.cc:492] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Bootstrap starting.
I20260807 22:42:43.867480 29844 tablet_bootstrap.cc:654] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:43.868403 29844 log.cc:824] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:43.868520 29845 tablet_bootstrap.cc:654] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:43.869266 29845 log.cc:824] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:43.870641 29844 tablet_bootstrap.cc:492] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: No bootstrap required, opened a new log
I20260807 22:42:43.870842 29844 ts_tablet_manager.cc:1397] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:42:43.871482 29845 tablet_bootstrap.cc:492] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: No bootstrap required, opened a new log
I20260807 22:42:43.871665 29845 ts_tablet_manager.cc:1397] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:42:43.872313 29850 tablet_bootstrap.cc:492] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Bootstrap starting.
I20260807 22:42:43.874634 29850 tablet_bootstrap.cc:654] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:43.874782 29844 raft_consensus.cc:348] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.875111 29844 raft_consensus.cc:374] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:43.875186 29844 raft_consensus.cc:729] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9d0f41d4a748658af23acdc5798658, State: Initialized, Role: FOLLOWER
I20260807 22:42:43.875418 29850 log.cc:824] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:43.875528 29845 raft_consensus.cc:348] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.875633 29844 consensus_queue.cc:260] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.875764 29845 raft_consensus.cc:374] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:43.875829 29845 raft_consensus.cc:729] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9090f8d1fc34b4abdc143e1fc3d3196, State: Initialized, Role: FOLLOWER
I20260807 22:42:43.876299 29369 heartbeater.cc:503] Master 127.26.189.254:34985 was elected leader, sending a full tablet report...
I20260807 22:42:43.876289 29845 consensus_queue.cc:260] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.876641 29844 ts_tablet_manager.cc:1428] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:42:43.876955 29587 heartbeater.cc:503] Master 127.26.189.254:34985 was elected leader, sending a full tablet report...
I20260807 22:42:43.877977 29850 tablet_bootstrap.cc:492] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: No bootstrap required, opened a new log
I20260807 22:42:43.878159 29850 ts_tablet_manager.cc:1397] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 22:42:43.879223 29845 ts_tablet_manager.cc:1428] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 22:42:43.881675 29850 raft_consensus.cc:348] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.881865 29850 raft_consensus.cc:374] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:43.881924 29850 raft_consensus.cc:729] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce73b3505ce42d1b6e07b9763d41acb, State: Initialized, Role: FOLLOWER
I20260807 22:42:43.882264 29850 consensus_queue.cc:260] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.882851 29806 heartbeater.cc:503] Master 127.26.189.254:34985 was elected leader, sending a full tablet report...
I20260807 22:42:43.883026 29850 ts_tablet_manager.cc:1428] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260807 22:42:43.986030 29852 raft_consensus.cc:482] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:43.986253 29852 raft_consensus.cc:504] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.987404 29852 leader_election.cc:288] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683), 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:43.992197 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.988154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:47657 (local address 127.26.189.195:43879)
0807 22:42:43.988274 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.988278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.988375 (+ 97us) server_negotiation.cc:407] Connection header received
0807 22:42:43.988463 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.988466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.988503 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.988567 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:43.988787 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.989506 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.991013 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.991622 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.991735 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.991787 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.992005 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.992070 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:42:43.992197 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.987783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:56121 (local address 127.26.189.194:45683)
0807 22:42:43.987903 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:43.987906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:43.988000 (+ 94us) server_negotiation.cc:407] Connection header received
0807 22:42:43.988161 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:43.988165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:43.988219 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:43.988305 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:43.988596 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.989376 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.991023 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.991615 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.991735 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.991787 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:43.992005 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 22:42:43.992070 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:42:43.992270 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.987673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:45683 (local address 127.26.189.193:56121)
0807 22:42:43.987910 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.987925 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.988038 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.988340 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.988343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.988365 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:43.988495 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.988501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.989481 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.989483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.990908 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.990920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.991726 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.991730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.991860 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.991911 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.991985 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.992087 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":97,"thread_start_us":70,"threads_started":1}
I20260807 22:42:43.992267 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:43.988017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:43879 (local address 127.26.189.193:47657)
0807 22:42:43.988291 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:43.988304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:43.988386 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:43.988591 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:43.988593 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:43.988612 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:43.988696 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.988701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.989592 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.989595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.990908 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:43.990920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:43.991726 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:43.991730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:43.991882 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:43.991915 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:42:43.991985 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:42:43.992095 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":136,"thread_start_us":63,"threads_started":1}
I20260807 22:42:43.992986 29732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59750943d84e0f98faea937fbd6524" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" is_pre_election: true
I20260807 22:42:43.993114 29512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59750943d84e0f98faea937fbd6524" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" is_pre_election: true
I20260807 22:42:43.993341 29512 raft_consensus.cc:2454] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 0.
I20260807 22:42:43.993341 29732 raft_consensus.cc:2454] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 0.
I20260807 22:42:43.993916 29206 leader_election.cc:302] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce73b3505ce42d1b6e07b9763d41acb, 8f9d0f41d4a748658af23acdc5798658; no voters:
I20260807 22:42:43.994177 29852 raft_consensus.cc:2781] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:43.994259 29852 raft_consensus.cc:482] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:43.994333 29852 raft_consensus.cc:3037] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:43.995368 29852 raft_consensus.cc:504] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:43.995675 29852 leader_election.cc:288] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 election: Requested vote from peers a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683), 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:43.995980 29512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59750943d84e0f98faea937fbd6524" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196"
I20260807 22:42:43.996047 29732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d59750943d84e0f98faea937fbd6524" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce73b3505ce42d1b6e07b9763d41acb"
I20260807 22:42:43.996088 29512 raft_consensus.cc:3037] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:43.996178 29732 raft_consensus.cc:3037] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:43.997390 29512 raft_consensus.cc:2454] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 1.
I20260807 22:42:43.997479 29732 raft_consensus.cc:2454] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 1.
I20260807 22:42:43.997676 29206 leader_election.cc:302] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9d0f41d4a748658af23acdc5798658, a9090f8d1fc34b4abdc143e1fc3d3196; no voters:
I20260807 22:42:43.997834 29852 raft_consensus.cc:2781] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:43.998026 29852 raft_consensus.cc:686] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9d0f41d4a748658af23acdc5798658, State: Running, Role: LEADER
I20260807 22:42:43.998522 29852 consensus_queue.cc:237] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.001993 29063 catalog_manager.cc:5495] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f9d0f41d4a748658af23acdc5798658 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } health_report { overall_health: UNKNOWN } } }
W20260807 22:42:44.047132 29807 tablet.cc:2369] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:44.049865 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.046138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.26.189.193:42191)
0807 22:42:44.046280 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.046285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.046325 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:42:44.046500 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.046502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.046560 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.046661 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:44.046905 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.047617 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.048580 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.048810 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.048882 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.049174 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:44.049564 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:44.049655 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.049707 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.049758 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:42:44.055420 29732 raft_consensus.cc:1261] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Refusing update from remote peer 8f9d0f41d4a748658af23acdc5798658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:44.055420 29512 raft_consensus.cc:1261] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Refusing update from remote peer 8f9d0f41d4a748658af23acdc5798658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:44.056080 29858 consensus_queue.cc:1035] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:44.056267 29852 consensus_queue.cc:1035] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:44.064448 29864 mvcc.cc:205] Tried to move back new op lower bound from 7316039942363856896 to 7316039942141325312. Current Snapshot: MvccSnapshot[applied={T|T < 7316039942363856896}]
I20260807 22:42:44.075193 29064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:44.075358 29064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:44.077981 29064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: Sending DeleteTablet for 3 replicas of tablet 8d59750943d84e0f98faea937fbd6524
I20260807 22:42:44.078990 29275 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d59750943d84e0f98faea937fbd6524 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:44 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51392
I20260807 22:42:44.079020 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d59750943d84e0f98faea937fbd6524 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:44 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:56140
I20260807 22:42:44.079023 29480 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d59750943d84e0f98faea937fbd6524 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:44 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:41742
I20260807 22:42:44.079622 29879 tablet_replica.cc:330] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: stopping tablet replica
I20260807 22:42:44.079937 29879 raft_consensus.cc:2229] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:44.080363 29879 raft_consensus.cc:2258] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:44.080587 29064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:42:44.080787 29878 tablet_replica.cc:330] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: stopping tablet replica
I20260807 22:42:44.081073 29878 raft_consensus.cc:2229] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 1 LEADER]: Raft consensus shutting down.
W20260807 22:42:44.081310 29064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:44.081436 29878 raft_consensus.cc:2258] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:44.082123 29879 ts_tablet_manager.cc:1905] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:44.082779 29878 ts_tablet_manager.cc:1905] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:44.085035 29879 ts_tablet_manager.cc:1918] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:42:44.085229 29879 log.cc:1196] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/8d59750943d84e0f98faea937fbd6524
I20260807 22:42:44.085386 29878 ts_tablet_manager.cc:1918] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:42:44.085632 29879 ts_tablet_manager.cc:1939] T 8d59750943d84e0f98faea937fbd6524 P a9090f8d1fc34b4abdc143e1fc3d3196: Deleting consensus metadata
I20260807 22:42:44.085705 29880 tablet_replica.cc:330] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: stopping tablet replica
I20260807 22:42:44.086050 29880 raft_consensus.cc:2229] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:44.086702 29700 tablet_service.cc:1467] Processing CreateTablet for tablet a1a82e499061456384df59be8c08c801 (DEFAULT_TABLE table=test-table [id=cbf13373a2c64dd18e3e02e84ccb0838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:44.086820 29480 tablet_service.cc:1467] Processing CreateTablet for tablet a1a82e499061456384df59be8c08c801 (DEFAULT_TABLE table=test-table [id=cbf13373a2c64dd18e3e02e84ccb0838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:44.086977 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a82e499061456384df59be8c08c801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:44.087069 29480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a82e499061456384df59be8c08c801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:44.088763 29850 tablet_bootstrap.cc:492] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Bootstrap starting.
I20260807 22:42:44.089071 29880 raft_consensus.cc:2258] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:44.089156 29845 tablet_bootstrap.cc:492] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Bootstrap starting.
I20260807 22:42:44.089735 29850 tablet_bootstrap.cc:654] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:44.090032 29845 tablet_bootstrap.cc:654] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:44.090585 29880 ts_tablet_manager.cc:1905] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:44.090761 29878 log.cc:1196] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/8d59750943d84e0f98faea937fbd6524
I20260807 22:42:44.090749 29041 catalog_manager.cc:4841] TS a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683): tablet 8d59750943d84e0f98faea937fbd6524 (table test-table [id=cd8a295c5f6745108ba97cbbb919ab54]) successfully deleted
I20260807 22:42:44.091064 29878 ts_tablet_manager.cc:1939] T 8d59750943d84e0f98faea937fbd6524 P 8f9d0f41d4a748658af23acdc5798658: Deleting consensus metadata
I20260807 22:42:44.091251 29275 tablet_service.cc:1467] Processing CreateTablet for tablet a1a82e499061456384df59be8c08c801 (DEFAULT_TABLE table=test-table [id=cbf13373a2c64dd18e3e02e84ccb0838]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:44.091487 29275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a82e499061456384df59be8c08c801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:44.093729 29844 tablet_bootstrap.cc:492] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Bootstrap starting.
I20260807 22:42:44.093968 29880 ts_tablet_manager.cc:1918] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:42:44.094049 29880 log.cc:1196] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/8d59750943d84e0f98faea937fbd6524
I20260807 22:42:44.094317 29880 ts_tablet_manager.cc:1939] T 8d59750943d84e0f98faea937fbd6524 P 2ce73b3505ce42d1b6e07b9763d41acb: Deleting consensus metadata
I20260807 22:42:44.094453 29041 catalog_manager.cc:4841] TS 8f9d0f41d4a748658af23acdc5798658 (127.26.189.193:42191): tablet 8d59750943d84e0f98faea937fbd6524 (table test-table [id=cd8a295c5f6745108ba97cbbb919ab54]) successfully deleted
I20260807 22:42:44.094655 29844 tablet_bootstrap.cc:654] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:44.095345 29845 tablet_bootstrap.cc:492] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: No bootstrap required, opened a new log
I20260807 22:42:44.095429 29845 ts_tablet_manager.cc:1397] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 22:42:44.095470 29041 catalog_manager.cc:4841] TS 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879): tablet 8d59750943d84e0f98faea937fbd6524 (table test-table [id=cd8a295c5f6745108ba97cbbb919ab54]) successfully deleted
I20260807 22:42:44.095737 29844 tablet_bootstrap.cc:492] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: No bootstrap required, opened a new log
I20260807 22:42:44.095804 29844 ts_tablet_manager.cc:1397] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:44.095871 29845 raft_consensus.cc:348] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.095985 29845 raft_consensus.cc:374] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:44.096019 29845 raft_consensus.cc:729] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9090f8d1fc34b4abdc143e1fc3d3196, State: Initialized, Role: FOLLOWER
I20260807 22:42:44.096148 29845 consensus_queue.cc:260] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.096211 29844 raft_consensus.cc:348] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.096276 29850 tablet_bootstrap.cc:492] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: No bootstrap required, opened a new log
I20260807 22:42:44.096306 29844 raft_consensus.cc:374] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:44.096335 29844 raft_consensus.cc:729] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9d0f41d4a748658af23acdc5798658, State: Initialized, Role: FOLLOWER
I20260807 22:42:44.096346 29850 ts_tablet_manager.cc:1397] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260807 22:42:44.096418 29845 ts_tablet_manager.cc:1428] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:44.096453 29844 consensus_queue.cc:260] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.096683 29844 ts_tablet_manager.cc:1428] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:44.096751 29850 raft_consensus.cc:348] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.096845 29850 raft_consensus.cc:374] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:44.096873 29850 raft_consensus.cc:729] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce73b3505ce42d1b6e07b9763d41acb, State: Initialized, Role: FOLLOWER
I20260807 22:42:44.096976 29850 consensus_queue.cc:260] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.097201 29850 ts_tablet_manager.cc:1428] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 22:42:44.125174 29373 tablet.cc:2369] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:44.260461 29872 raft_consensus.cc:482] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:44.260608 29872 raft_consensus.cc:504] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.261050 29872 leader_election.cc:288] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683), 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:44.261497 29512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a82e499061456384df59be8c08c801" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" is_pre_election: true
I20260807 22:42:44.261562 29732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a82e499061456384df59be8c08c801" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" is_pre_election: true
I20260807 22:42:44.261659 29512 raft_consensus.cc:2454] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 0.
I20260807 22:42:44.261739 29732 raft_consensus.cc:2454] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 0.
I20260807 22:42:44.261960 29206 leader_election.cc:302] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9d0f41d4a748658af23acdc5798658, a9090f8d1fc34b4abdc143e1fc3d3196; no voters:
I20260807 22:42:44.262136 29872 raft_consensus.cc:2781] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:44.262198 29872 raft_consensus.cc:482] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:44.262243 29872 raft_consensus.cc:3037] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:44.263347 29872 raft_consensus.cc:504] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.263636 29872 leader_election.cc:288] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 election: Requested vote from peers a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683), 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:44.264036 29732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a82e499061456384df59be8c08c801" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce73b3505ce42d1b6e07b9763d41acb"
I20260807 22:42:44.264039 29512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a82e499061456384df59be8c08c801" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196"
I20260807 22:42:44.264159 29732 raft_consensus.cc:3037] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:44.264168 29512 raft_consensus.cc:3037] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:44.265115 29512 raft_consensus.cc:2454] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 1.
I20260807 22:42:44.265197 29732 raft_consensus.cc:2454] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 1.
I20260807 22:42:44.265445 29206 leader_election.cc:302] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9d0f41d4a748658af23acdc5798658, a9090f8d1fc34b4abdc143e1fc3d3196; no voters:
I20260807 22:42:44.265609 29872 raft_consensus.cc:2781] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:44.265700 29872 raft_consensus.cc:686] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9d0f41d4a748658af23acdc5798658, State: Running, Role: LEADER
I20260807 22:42:44.265839 29872 consensus_queue.cc:237] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.267546 29063 catalog_manager.cc:5495] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f9d0f41d4a748658af23acdc5798658 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:42:44 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:44.315387 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.307837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:34985 (local address 127.0.0.1:32952)
0807 22:42:44.308134 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:44.308144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:44.308238 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:44.308437 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:44.308439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:44.308833 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:44.308928 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.308934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.309726 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.309728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.310261 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.310267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.310649 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.310651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.310708 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.311197 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:44.311208 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:44.312548 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:44.314099 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:44.314105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:44.314110 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:44.314349 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:44.314576 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:44.314578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:44.314579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:44.314656 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:44.314936 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:44.314941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:44.315036 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 22:42:44.315114 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:42:44.315231 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":96,"threads_started":1}
I20260807 22:42:44.315408 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.307891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.26.189.254:34985)
0807 22:42:44.308011 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.308014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.308212 (+ 198us) server_negotiation.cc:407] Connection header received
0807 22:42:44.308306 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.308308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.308349 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.308414 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:44.309049 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.309642 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.310353 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.310567 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.310642 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.312678 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:44.312681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:44.312683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:44.312705 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:44.314017 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.314430 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.314432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.314433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.314511 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.314715 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.314716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.314718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.314867 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:44.314945 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.315156 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.315288 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":43}
I20260807 22:42:44.318907 29063 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:32952:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:42:44.319662 29063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:44.324127 29275 tablet_service.cc:1467] Processing CreateTablet for tablet 798438ccebfe4acdb560bfd3bd4e2ee3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2850f3e2208b4d1dbb39a9c6c5591a61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:44.324153 29480 tablet_service.cc:1467] Processing CreateTablet for tablet 798438ccebfe4acdb560bfd3bd4e2ee3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2850f3e2208b4d1dbb39a9c6c5591a61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:44.324276 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 798438ccebfe4acdb560bfd3bd4e2ee3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2850f3e2208b4d1dbb39a9c6c5591a61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:44.324545 29275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 798438ccebfe4acdb560bfd3bd4e2ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:44.324553 29480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 798438ccebfe4acdb560bfd3bd4e2ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:44.324585 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 798438ccebfe4acdb560bfd3bd4e2ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:44.327240 29844 tablet_bootstrap.cc:492] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: Bootstrap starting.
I20260807 22:42:44.327386 29850 tablet_bootstrap.cc:492] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: Bootstrap starting.
I20260807 22:42:44.327987 29844 tablet_bootstrap.cc:654] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:44.328207 29850 tablet_bootstrap.cc:654] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:44.328876 29845 tablet_bootstrap.cc:492] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: Bootstrap starting.
I20260807 22:42:44.329341 29850 tablet_bootstrap.cc:492] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: No bootstrap required, opened a new log
I20260807 22:42:44.329416 29850 ts_tablet_manager.cc:1397] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:44.329463 29844 tablet_bootstrap.cc:492] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: No bootstrap required, opened a new log
I20260807 22:42:44.329531 29844 ts_tablet_manager.cc:1397] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:44.329623 29845 tablet_bootstrap.cc:654] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:44.329864 29844 raft_consensus.cc:348] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.329859 29850 raft_consensus.cc:348] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.329962 29850 raft_consensus.cc:374] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:44.329962 29844 raft_consensus.cc:374] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:44.329993 29850 raft_consensus.cc:729] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce73b3505ce42d1b6e07b9763d41acb, State: Initialized, Role: FOLLOWER
I20260807 22:42:44.329993 29844 raft_consensus.cc:729] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9d0f41d4a748658af23acdc5798658, State: Initialized, Role: FOLLOWER
I20260807 22:42:44.330104 29850 consensus_queue.cc:260] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.330116 29844 consensus_queue.cc:260] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.330379 29850 ts_tablet_manager.cc:1428] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:44.330387 29844 ts_tablet_manager.cc:1428] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:44.330536 29854 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.330580 29872 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.330679 29854 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:44.330696 29872 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:44.330725 29845 tablet_bootstrap.cc:492] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: No bootstrap required, opened a new log
I20260807 22:42:44.330793 29845 ts_tablet_manager.cc:1397] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:44.331198 29845 raft_consensus.cc:348] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.331311 29845 raft_consensus.cc:374] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:44.331346 29845 raft_consensus.cc:729] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9090f8d1fc34b4abdc143e1fc3d3196, State: Initialized, Role: FOLLOWER
I20260807 22:42:44.331446 29845 consensus_queue.cc:260] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.331681 29845 ts_tablet_manager.cc:1428] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:42:44.331701 29853 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.331836 29853 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 22:42:44.339938 29588 tablet.cc:2369] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:44.383109 29872 raft_consensus.cc:482] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:44.383217 29872 raft_consensus.cc:504] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.383572 29872 leader_election.cc:288] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683), 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:44.383994 29732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798438ccebfe4acdb560bfd3bd4e2ee3" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" is_pre_election: true
I20260807 22:42:44.384008 29512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798438ccebfe4acdb560bfd3bd4e2ee3" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" is_pre_election: true
I20260807 22:42:44.384141 29732 raft_consensus.cc:2454] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 0.
I20260807 22:42:44.384156 29512 raft_consensus.cc:2454] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 0.
I20260807 22:42:44.384416 29206 leader_election.cc:302] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce73b3505ce42d1b6e07b9763d41acb, 8f9d0f41d4a748658af23acdc5798658; no voters:
I20260807 22:42:44.384585 29872 raft_consensus.cc:2781] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:44.384653 29872 raft_consensus.cc:482] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:44.384697 29872 raft_consensus.cc:3037] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:44.385636 29872 raft_consensus.cc:504] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.385914 29872 leader_election.cc:288] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 election: Requested vote from peers a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683), 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879)
I20260807 22:42:44.386260 29512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798438ccebfe4acdb560bfd3bd4e2ee3" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196"
I20260807 22:42:44.386260 29732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798438ccebfe4acdb560bfd3bd4e2ee3" candidate_uuid: "8f9d0f41d4a748658af23acdc5798658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce73b3505ce42d1b6e07b9763d41acb"
I20260807 22:42:44.386392 29512 raft_consensus.cc:3037] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:44.386420 29732 raft_consensus.cc:3037] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:44.387244 29732 raft_consensus.cc:2454] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 1.
I20260807 22:42:44.387501 29206 leader_election.cc:302] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce73b3505ce42d1b6e07b9763d41acb, 8f9d0f41d4a748658af23acdc5798658; no voters:
I20260807 22:42:44.387624 29512 raft_consensus.cc:2454] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9d0f41d4a748658af23acdc5798658 in term 1.
I20260807 22:42:44.387748 29872 raft_consensus.cc:2781] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:44.387835 29872 raft_consensus.cc:686] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9d0f41d4a748658af23acdc5798658, State: Running, Role: LEADER
I20260807 22:42:44.387986 29872 consensus_queue.cc:237] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } }
I20260807 22:42:44.388628 29858 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: TxnStatusTablet state changed. Reason: New leader 8f9d0f41d4a748658af23acdc5798658. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.388758 29858 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:44.389429 29902 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:42:44.389612 29902 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:42:44.389858 29062 catalog_manager.cc:5495] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f9d0f41d4a748658af23acdc5798658 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } health_report { overall_health: UNKNOWN } } }
I20260807 22:42:44.412020 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.408437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.26.189.193:42191)
0807 22:42:44.408547 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.408551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.408676 (+ 125us) server_negotiation.cc:407] Connection header received
0807 22:42:44.408829 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.408831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.408875 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.408936 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:44.409176 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.409680 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.410909 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.411139 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.411230 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.411576 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:44.411662 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:44.411737 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.411838 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.411895 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:42:44.412045 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.408360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42191 (local address 127.0.0.1:51422)
0807 22:42:44.408606 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:44.408621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:44.408733 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:44.408968 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:44.408971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:44.408979 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:44.409089 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.409095 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.409768 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.409772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.410779 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.410789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.411242 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.411246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.411333 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.411386 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:44.411749 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:44.411761 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 22:42:44.411823 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 22:42:44.411908 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260807 22:42:44.414887 29512 raft_consensus.cc:1261] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Refusing update from remote peer 8f9d0f41d4a748658af23acdc5798658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:44.414893 29732 raft_consensus.cc:1261] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Refusing update from remote peer 8f9d0f41d4a748658af23acdc5798658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:44.415302 29872 consensus_queue.cc:1035] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:44.415467 29858 consensus_queue.cc:1035] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:44.416848 29854 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: TxnStatusTablet state changed. Reason: New leader 8f9d0f41d4a748658af23acdc5798658. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.416965 29854 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:44.416997 29853 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: TxnStatusTablet state changed. Reason: New leader 8f9d0f41d4a748658af23acdc5798658. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.417104 29853 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:44.419178 29872 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.419281 29872 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:44.419389 29872 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.419464 29872 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 8f9d0f41d4a748658af23acdc5798658: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:44.419528 29863 mvcc.cc:205] Tried to move back new op lower bound from 7316039943839047680 to 7316039943734677504. Current Snapshot: MvccSnapshot[applied={T|T < 7316039943839047680}]
I20260807 22:42:44.420624 29853 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.420720 29853 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P a9090f8d1fc34b4abdc143e1fc3d3196: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:44.421406 29854 tablet_replica.cc:439] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f9d0f41d4a748658af23acdc5798658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f9d0f41d4a748658af23acdc5798658" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42191 } } peers { permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 } } peers { permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 } } }
I20260807 22:42:44.421525 29854 tablet_replica.cc:442] T 798438ccebfe4acdb560bfd3bd4e2ee3 P 2ce73b3505ce42d1b6e07b9763d41acb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:44.434432 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.431386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:51619 (local address 127.26.189.193:42191)
0807 22:42:44.431501 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.431506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.431723 (+ 217us) server_negotiation.cc:407] Connection header received
0807 22:42:44.431853 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.431856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.431894 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.431955 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:44.432162 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.432648 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.433441 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.433648 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.433732 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.434005 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:44.434068 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:44.434127 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.434218 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.434262 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":46}
I20260807 22:42:44.434432 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.431313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42191 (local address 127.26.189.193:51619)
0807 22:42:44.431645 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:44.431658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:44.431757 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:44.431974 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:44.431976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:44.431983 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:44.432067 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.432072 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.432747 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.432750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.433339 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:44.433347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.433746 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.433750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:44.433856 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.433893 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:44.434142 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:44.434148 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:42:44.434201 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 22:42:44.434276 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":17,"thread_start_us":89,"threads_started":1}
I20260807 22:42:44.440582 29732 raft_consensus.cc:1261] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Refusing update from remote peer 8f9d0f41d4a748658af23acdc5798658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:44.440665 29512 raft_consensus.cc:1261] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Refusing update from remote peer 8f9d0f41d4a748658af23acdc5798658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:44.440940 29872 consensus_queue.cc:1035] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce73b3505ce42d1b6e07b9763d41acb" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:44.441206 29872 consensus_queue.cc:1035] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9090f8d1fc34b4abdc143e1fc3d3196" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:44.446053 29865 mvcc.cc:205] Tried to move back new op lower bound from 7316039943945723904 to 7316039943234240512. Current Snapshot: MvccSnapshot[applied={T|T < 7316039943945723904}]
I20260807 22:42:44.478121 29062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:44.478276 29062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:44.480281 29062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 980200fbe79044c88043caed8d8890c7: Sending DeleteTablet for 3 replicas of tablet a1a82e499061456384df59be8c08c801
I20260807 22:42:44.480914 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a82e499061456384df59be8c08c801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:44 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:56140
I20260807 22:42:44.480921 29480 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a82e499061456384df59be8c08c801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:44 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:41742
I20260807 22:42:44.481096 29879 tablet_replica.cc:330] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: stopping tablet replica
I20260807 22:42:44.481108 29880 tablet_replica.cc:330] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: stopping tablet replica
I20260807 22:42:44.481208 29879 raft_consensus.cc:2229] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:44.481226 29880 raft_consensus.cc:2229] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:44.481302 29879 raft_consensus.cc:2258] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:44.481325 29880 raft_consensus.cc:2258] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:44.481956 29879 ts_tablet_manager.cc:1905] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:44.481937 29275 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a82e499061456384df59be8c08c801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:44 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51392
I20260807 22:42:44.482105 29878 tablet_replica.cc:330] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: stopping tablet replica
I20260807 22:42:44.482223 29878 raft_consensus.cc:2229] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:42:44.482424 29878 raft_consensus.cc:2258] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:44.483145 29878 ts_tablet_manager.cc:1905] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:44.484848 29879 ts_tablet_manager.cc:1918] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:44.484932 29879 log.cc:1196] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/a1a82e499061456384df59be8c08c801
I20260807 22:42:44.485184 29879 ts_tablet_manager.cc:1939] T a1a82e499061456384df59be8c08c801 P a9090f8d1fc34b4abdc143e1fc3d3196: Deleting consensus metadata
I20260807 22:42:44.485872 29878 ts_tablet_manager.cc:1918] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:44.485855 29041 catalog_manager.cc:4841] TS a9090f8d1fc34b4abdc143e1fc3d3196 (127.26.189.194:45683): tablet a1a82e499061456384df59be8c08c801 (table test-table [id=cbf13373a2c64dd18e3e02e84ccb0838]) successfully deleted
I20260807 22:42:44.486039 29878 log.cc:1196] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/a1a82e499061456384df59be8c08c801
I20260807 22:42:44.486334 29878 ts_tablet_manager.cc:1939] T a1a82e499061456384df59be8c08c801 P 8f9d0f41d4a748658af23acdc5798658: Deleting consensus metadata
I20260807 22:42:44.486403 29880 ts_tablet_manager.cc:1905] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:44.487001 29041 catalog_manager.cc:4841] TS 8f9d0f41d4a748658af23acdc5798658 (127.26.189.193:42191): tablet a1a82e499061456384df59be8c08c801 (table test-table [id=cbf13373a2c64dd18e3e02e84ccb0838]) successfully deleted
I20260807 22:42:44.488540 29880 ts_tablet_manager.cc:1918] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:44.488618 29880 log.cc:1196] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/a1a82e499061456384df59be8c08c801
I20260807 22:42:44.488816 29880 ts_tablet_manager.cc:1939] T a1a82e499061456384df59be8c08c801 P 2ce73b3505ce42d1b6e07b9763d41acb: Deleting consensus metadata
I20260807 22:42:44.489442 29041 catalog_manager.cc:4841] TS 2ce73b3505ce42d1b6e07b9763d41acb (127.26.189.195:43879): tablet a1a82e499061456384df59be8c08c801 (table test-table [id=cbf13373a2c64dd18e3e02e84ccb0838]) successfully deleted
Aug 07 22:42:44 dist-test-slave-4tdj krb5kdc[28999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:44.495934 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.482396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.26.189.193:42191)
0807 22:42:44.482519 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.482524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.486413 (+ 3889us) server_negotiation.cc:407] Connection header received
0807 22:42:44.486618 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.486622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.486669 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.486719 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:44.487599 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.488091 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.488982 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.489213 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.489299 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.492229 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:44.492241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:44.492257 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:44.492279 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:44.493993 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.494507 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.494513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.494515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.494638 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.494961 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.494965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.494967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.495278 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:44.495467 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.495661 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.495786 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 22:42:44.496304 29362 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51426
I20260807 22:42:44.505155 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:44.497874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.26.189.254:34985)
0807 22:42:44.497990 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:44.497994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:44.498101 (+ 107us) server_negotiation.cc:407] Connection header received
0807 22:42:44.498211 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:44.498213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:44.498252 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:44.498377 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:44.499028 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.499518 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.500516 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:44.500780 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:44.500852 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:44.502516 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:44.502519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:44.502521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:44.502543 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:44.503775 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.504182 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.504184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.504185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.504250 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:44.504466 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:44.504467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:44.504469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:44.504631 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:44.504723 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:44.504956 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 22:42:44.505040 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 22:42:44.505518 29062 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32960
I20260807 22:42:44.506776 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29181
I20260807 22:42:44.536908 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29382
I20260807 22:42:44.548008 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29606
I20260807 22:42:44.557833 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29018
2026-08-07T22:42:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2879 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:44 dist-test-slave-4tdj krb5kdc[29931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:44 dist-test-slave-4tdj krb5kdc[29931](info): set up 2 sockets
Aug 07 22:42:44 dist-test-slave-4tdj krb5kdc[29931](info): commencing operation
krb5kdc: starting...
W20260807 22:42:45.690722 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:45 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:45.806675 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38853
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:45.913681 29949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:45.913923 29949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:45.913959 29949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:45.917793 29949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:45.917841 29949 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:42:45.917856 29949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:45.917872 29949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:45.917893 29949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:45.917907 29949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:45.922974 29949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38853
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:45.923884 29949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:45.924803 29949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:45.933084 29958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:45.933185 29957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:45.933259 29949 server_base.cc:1034] running on GCE node
W20260807 22:42:45.933100 29961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:45.934082 29949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:45.935148 29949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:45.936295 29949 hybrid_clock.cc:648] HybridClock initialized: now 1786142565936277 us; error 32 us; skew 500 ppm
Aug 07 22:42:45 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:45.938889 29949 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:45.939764 29949 webserver.cc:460] Webserver started at http://127.26.189.254:33001/ using document root <none> and password file <none>
I20260807 22:42:45.940276 29949 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:45.940340 29949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:45.940548 29949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:45.942644 29949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "3169bfbdb12f4c3899998b21c118acad"
format_stamp: "Formatted at 2026-08-07 22:42:45 on dist-test-slave-4tdj"
server_key: "0a94c467b60ef9123defe8ca4feb2917"
server_key_iv: "13de7e4d7e3c7fea5855078d9dd10a1a"
server_key_version: "encryptionkey@0"
I20260807 22:42:45.943179 29949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "3169bfbdb12f4c3899998b21c118acad"
format_stamp: "Formatted at 2026-08-07 22:42:45 on dist-test-slave-4tdj"
server_key: "0a94c467b60ef9123defe8ca4feb2917"
server_key_iv: "13de7e4d7e3c7fea5855078d9dd10a1a"
server_key_version: "encryptionkey@0"
I20260807 22:42:45.946779 29949 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:45.949100 29972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:45.950240 29949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:45.950382 29949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "3169bfbdb12f4c3899998b21c118acad"
format_stamp: "Formatted at 2026-08-07 22:42:45 on dist-test-slave-4tdj"
server_key: "0a94c467b60ef9123defe8ca4feb2917"
server_key_iv: "13de7e4d7e3c7fea5855078d9dd10a1a"
server_key_version: "encryptionkey@0"
I20260807 22:42:45.950480 29949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:45.962014 29949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:45.964684 29949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:45.964846 29949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:45.972333 29949 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38853
I20260807 22:42:45.972362 30079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38853 every 8 connection(s)
I20260807 22:42:45.973227 29949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:45.975885 30080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:45.981336 30080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Bootstrap starting.
I20260807 22:42:45.982329 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29949
I20260807 22:42:45.982429 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:45.982657 27383 external_mini_cluster.cc:1428] Setting key 20beee4d9c24d33817c5c2e065c1033d
I20260807 22:42:45.983979 30080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:45.984826 30080 log.cc:824] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:45.986950 30080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: No bootstrap required, opened a new log
I20260807 22:42:45.990870 30080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3169bfbdb12f4c3899998b21c118acad" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38853 } }
I20260807 22:42:45.991142 30080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:45.991202 30080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3169bfbdb12f4c3899998b21c118acad, State: Initialized, Role: FOLLOWER
Aug 07 22:42:45 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:45.991676 30080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3169bfbdb12f4c3899998b21c118acad" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38853 } }
I20260807 22:42:45.991833 30080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:45.991907 30080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:45.992003 30080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:45.993041 30080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3169bfbdb12f4c3899998b21c118acad" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38853 } }
I20260807 22:42:45.993363 30080 leader_election.cc:302] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3169bfbdb12f4c3899998b21c118acad; no voters:
I20260807 22:42:45.993624 30080 leader_election.cc:288] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:45.993736 30085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:45.993953 30085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [term 1 LEADER]: Becoming Leader. State: Replica: 3169bfbdb12f4c3899998b21c118acad, State: Running, Role: LEADER
I20260807 22:42:45.994338 30085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3169bfbdb12f4c3899998b21c118acad" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38853 } }
I20260807 22:42:45.994874 30080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:45.996016 30086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3169bfbdb12f4c3899998b21c118acad. Latest consensus state: current_term: 1 leader_uuid: "3169bfbdb12f4c3899998b21c118acad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3169bfbdb12f4c3899998b21c118acad" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38853 } } }
I20260807 22:42:45.996124 30087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3169bfbdb12f4c3899998b21c118acad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3169bfbdb12f4c3899998b21c118acad" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38853 } } }
I20260807 22:42:45.996215 30086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:45.996223 30087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:45.996717 30099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:45.996984 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:45.983648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37940 (local address 127.26.189.254:38853)
0807 22:42:45.984086 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:45.984106 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:45.984149 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:42:45.984970 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:45.984993 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:45.985390 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:45.985712 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:45.986348 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.987782 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.988603 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:45.989284 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:45.989442 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:45.991970 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:45.991977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:45.991994 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:45.992024 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:45.993697 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.994229 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.994235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.994241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.994412 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:45.994739 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:45.994743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:45.994745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:45.995245 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:45.995469 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:45.995721 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 22:42:45.996015 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":129,"threads_started":1}
I20260807 22:42:45.999814 30099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:46.004958 30099 catalog_manager.cc:1347] Generated new cluster ID: cbe29243916f49308ba09610125ca917
I20260807 22:42:46.005026 30099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:46.016513 30099 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:46.017655 30099 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:46.021903 30099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Generated new TSK 0
I20260807 22:42:46.022643 30099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:46.056988 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38853
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:46.165545 30117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:46.165779 30117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:46.165841 30117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:46.169698 30117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:46.169747 30117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:46.169800 30117 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:42:46.169828 30117 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:46.169873 30117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:46.174790 30117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:38853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:46.175709 30117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:46.176632 30117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:46.185518 30125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:46.185526 30128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:46.185526 30130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:46.185770 30117 server_base.cc:1034] running on GCE node
I20260807 22:42:46.186131 30117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:46.186769 30117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:46.187927 30117 hybrid_clock.cc:648] HybridClock initialized: now 1786142566187914 us; error 27 us; skew 500 ppm
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:46.190449 30117 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:46.191275 30117 webserver.cc:460] Webserver started at http://127.26.189.193:39599/ using document root <none> and password file <none>
I20260807 22:42:46.191785 30117 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:46.191848 30117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:46.192029 30117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:46.193902 30117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "a88df149af36419094c2429ce10f30f6"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "ebd309608afb47f058263c381f8a7cda"
server_key_iv: "f7bda4ee53ac630b3062fcdfa9c1af99"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.194470 30117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "a88df149af36419094c2429ce10f30f6"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "ebd309608afb47f058263c381f8a7cda"
server_key_iv: "f7bda4ee53ac630b3062fcdfa9c1af99"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.197932 30117 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:46.200176 30145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:46.201247 30117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:46.201349 30117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "a88df149af36419094c2429ce10f30f6"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "ebd309608afb47f058263c381f8a7cda"
server_key_iv: "f7bda4ee53ac630b3062fcdfa9c1af99"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.201434 30117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:46.215996 30117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:46.218695 30117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:46.218858 30117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:46.220193 30117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:46.220242 30117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:46.220293 30117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:46.220322 30117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:46.230587 30117 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:43939
I20260807 22:42:46.230593 30331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:43939 every 8 connection(s)
I20260807 22:42:46.231597 30117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:46.233255 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 30117
I20260807 22:42:46.233358 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:46.233599 27383 external_mini_cluster.cc:1428] Setting key c1f9234aa0d16dda720c161235a056f0
I20260807 22:42:46.234998 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.220867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:43075 (local address 127.26.189.254:38853)
0807 22:42:46.220974 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.220977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.222648 (+ 1671us) server_negotiation.cc:407] Connection header received
0807 22:42:46.224054 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.224058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.224122 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.224231 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.225672 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.226510 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.227996 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.228378 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.228511 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.231299 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.231302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.231304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.231328 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.232796 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.233324 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.233328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.233330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.233441 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.233805 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.233808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.233810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.234043 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.234153 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.234676 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.234811 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:42:46.235741 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.221886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.26.189.193:43075)
0807 22:42:46.222489 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.222534 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.223805 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.224408 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.224416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.224867 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.225491 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.225511 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.226655 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.226659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.227844 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.227855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.228533 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.228537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.228736 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.229329 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.229350 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.231133 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.232903 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.232907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.232916 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.233217 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.233546 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.233550 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.233552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.233704 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.234169 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.234174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.234450 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.234600 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.234851 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":145,"threads_started":1}
I20260807 22:42:46.237030 30333 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38853
I20260807 22:42:46.237219 30333 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:46.237715 30333 heartbeater.cc:511] Master 127.26.189.254:38853 requested a full tablet report, sending...
I20260807 22:42:46.239323 29999 ts_manager.cc:194] Registered new tserver with Master: a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939)
I20260807 22:42:46.240751 29999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:43075
I20260807 22:42:46.246665 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.239006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.26.189.193:37311)
0807 22:42:46.239286 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.239297 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.239362 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.240130 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.240132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.240281 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.240369 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.240374 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.241286 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.241290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.242076 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.242087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.242700 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.242703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.242806 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.243245 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.243255 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.243551 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.245330 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.245332 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.245334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.245512 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.245796 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.245798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.245799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.245862 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.246250 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.246252 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.246347 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.246426 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.246521 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
I20260807 22:42:46.246799 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.239071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37311 (local address 127.26.189.254:38853)
0807 22:42:46.239920 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.239925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.239941 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:42:46.239978 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.239981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.240023 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.240115 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.240477 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.241152 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.242211 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.242572 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.242692 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.243692 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.243696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.243700 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.243728 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.245224 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.245603 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.245607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.245608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.245704 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.245940 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.245943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.245945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.246162 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.246274 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.246518 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.246631 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773}
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:46.263046 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38853
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:46.368232 30350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:46.368474 30350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:46.368520 30350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:46.372476 30350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:46.372531 30350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:46.372570 30350 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:42:46.372593 30350 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:46.372624 30350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:46.377630 30350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:38853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:46.378608 30350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:46.379544 30350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:46.388671 30364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:46.388669 30357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:46.388665 30358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:46.388957 30350 server_base.cc:1034] running on GCE node
I20260807 22:42:46.389312 30350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:46.389873 30350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:46.391031 30350 hybrid_clock.cc:648] HybridClock initialized: now 1786142566391009 us; error 27 us; skew 500 ppm
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:46.393483 30350 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:46.394265 30350 webserver.cc:460] Webserver started at http://127.26.189.194:41419/ using document root <none> and password file <none>
I20260807 22:42:46.394846 30350 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:46.394914 30350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:46.395105 30350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:46.397061 30350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "d9a13898824148e98fb6c55d6c487a64"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "fd7eec80e826eaf375e13e15fd422582"
server_key_iv: "c45f13e4ee490cdcac938a176ea61d42"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.397626 30350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "d9a13898824148e98fb6c55d6c487a64"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "fd7eec80e826eaf375e13e15fd422582"
server_key_iv: "c45f13e4ee490cdcac938a176ea61d42"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.401067 30350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:42:46.403301 30371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:46.404408 30350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:46.404511 30350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "d9a13898824148e98fb6c55d6c487a64"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "fd7eec80e826eaf375e13e15fd422582"
server_key_iv: "c45f13e4ee490cdcac938a176ea61d42"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.404601 30350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:46.441423 30350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:46.444212 30350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:46.444379 30350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:46.445823 30350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:46.445874 30350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:46.445928 30350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:46.445953 30350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:46.455852 30350 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:34771
I20260807 22:42:46.455928 30568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:34771 every 8 connection(s)
I20260807 22:42:46.456907 30350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:46.458446 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 30350
I20260807 22:42:46.458541 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:46.458861 27383 external_mini_cluster.cc:1428] Setting key d754c6aac20cc0d95fcb143fd7680fa8
I20260807 22:42:46.459597 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.446584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:35235 (local address 127.26.189.254:38853)
0807 22:42:46.446732 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.446738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.447551 (+ 813us) server_negotiation.cc:407] Connection header received
0807 22:42:46.448551 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.448555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.448621 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.448679 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.450306 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.450911 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.452120 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.452458 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.452575 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.455981 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.455985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.455988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.456016 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.457542 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.458077 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.458080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.458082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.458172 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.458567 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.458570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.458572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.458799 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.458901 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.459328 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.459445 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:42:46.460439 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.446958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.26.189.194:35235)
0807 22:42:46.447395 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.447445 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.448319 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.449096 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.449106 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.449545 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.450142 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.450163 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.451037 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.451040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.451990 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.452000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.452638 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.452644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.452852 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.454069 (+ 1217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.454091 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.455794 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.457658 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.457664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.457677 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.457983 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.458311 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.458316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.458318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.458467 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.458893 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.458898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.459103 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.459276 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.459557 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":96,"threads_started":1}
I20260807 22:42:46.461539 30571 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38853
I20260807 22:42:46.461686 30571 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:46.462138 30571 heartbeater.cc:511] Master 127.26.189.254:38853 requested a full tablet report, sending...
I20260807 22:42:46.463151 29999 ts_manager.cc:194] Registered new tserver with Master: d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771)
I20260807 22:42:46.463724 29999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:35235
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:42:46.470515 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.463674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:43949 (local address 127.26.189.254:38853)
0807 22:42:46.463781 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.463785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.464090 (+ 305us) server_negotiation.cc:407] Connection header received
0807 22:42:46.464247 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.464250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.464290 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.464333 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.464982 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.465526 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.466331 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.466680 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.466787 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.467796 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.467800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.467802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.467825 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.469026 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.469397 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.469400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.469402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.469475 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.469725 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.469728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.469730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.469928 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.470012 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.470253 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.470374 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:42:46.470530 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.463598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.26.189.194:43949)
0807 22:42:46.464010 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.464025 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.464152 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.464453 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.464455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.464667 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.464851 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.464860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.465664 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.465666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.466198 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.466204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.466788 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.466791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.466865 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.467362 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.467373 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.467674 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.469129 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.469132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.469134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.469312 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.469553 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.469556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.469558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.469646 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.470019 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.470022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.470108 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.470192 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.470357 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":1,"thread_start_us":234,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:46.487044 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38853
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:46.595181 30588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:46.595458 30588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:46.595526 30588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:46.599470 30588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:46.599521 30588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:46.599575 30588 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:42:46.599609 30588 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:42:46.599663 30588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:46.604702 30588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:38853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:46.605693 30588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:46.606659 30588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:46.615705 30600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:46.615725 30595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:46.615773 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:46.615890 30588 server_base.cc:1034] running on GCE node
I20260807 22:42:46.616344 30588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:46.616906 30588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:46.618060 30588 hybrid_clock.cc:648] HybridClock initialized: now 1786142566618050 us; error 28 us; skew 500 ppm
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:46.620687 30588 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:46.621456 30588 webserver.cc:460] Webserver started at http://127.26.189.195:34485/ using document root <none> and password file <none>
I20260807 22:42:46.621958 30588 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:46.622022 30588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:46.622221 30588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:46.624112 30588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "90a1f275c6b44657ae66f48ca7021be1"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "6e8e3e86c9a1d792b78279f0c05b2b68"
server_key_iv: "bea070f4bc41de07cae613dac90f0714"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.624629 30588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "90a1f275c6b44657ae66f48ca7021be1"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "6e8e3e86c9a1d792b78279f0c05b2b68"
server_key_iv: "bea070f4bc41de07cae613dac90f0714"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.628044 30588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:42:46.630198 30611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:46.631336 30588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:46.631438 30588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "90a1f275c6b44657ae66f48ca7021be1"
format_stamp: "Formatted at 2026-08-07 22:42:46 on dist-test-slave-4tdj"
server_key: "6e8e3e86c9a1d792b78279f0c05b2b68"
server_key_iv: "bea070f4bc41de07cae613dac90f0714"
server_key_version: "encryptionkey@0"
I20260807 22:42:46.631524 30588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:46.645368 30588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:46.648224 30588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:46.648388 30588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:46.649747 30588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:46.649797 30588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:46.649850 30588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:46.649875 30588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:46.659946 30588 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:36875
I20260807 22:42:46.660012 30802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:36875 every 8 connection(s)
I20260807 22:42:46.660776 30588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:46.662137 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.650457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:45633 (local address 127.26.189.254:38853)
0807 22:42:46.650569 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.650573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.651566 (+ 993us) server_negotiation.cc:407] Connection header received
0807 22:42:46.652550 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.652554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.652614 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.652668 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.654191 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.654863 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.656019 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.656260 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.656360 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.658729 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.658733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.658736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.658762 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.660151 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.660595 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.660599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.660601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.660679 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.661036 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.661040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.661042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.661248 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.661340 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.661916 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.662018 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:42:46.662511 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 30588
I20260807 22:42:46.662613 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:46.662864 27383 external_mini_cluster.cc:1428] Setting key 44a414ace38bfdb89da853daea710142
I20260807 22:42:46.662899 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.650952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.26.189.195:45633)
0807 22:42:46.651428 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.651467 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.652310 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.653052 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.653061 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.653436 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.654043 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.654065 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.654979 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.654982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.655896 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.655906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.656416 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.656420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.656584 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.657103 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.657123 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.658583 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.660246 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.660252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.660265 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.660495 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.660760 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.660762 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.660764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.660947 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.661395 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.661401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.661670 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.661842 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.662074 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":142,"threads_started":1}
I20260807 22:42:46.664156 30805 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38853
I20260807 22:42:46.664314 30805 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:46.664832 30805 heartbeater.cc:511] Master 127.26.189.254:38853 requested a full tablet report, sending...
I20260807 22:42:46.665791 29998 ts_manager.cc:194] Registered new tserver with Master: 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:46.666504 29998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:45633
I20260807 22:42:46.672102 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.666105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.26.189.195:54753)
0807 22:42:46.666357 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.666368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.666443 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.666671 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.666674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.666850 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.666956 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.666962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.667645 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.667649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.668199 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.668207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.668606 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.668610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.668697 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.669076 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.669090 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.669351 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.670815 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.670818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.670820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.670996 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.671238 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.671241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.671243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.671322 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.671641 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.671645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.671728 (+ 83us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.671811 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.671934 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260807 22:42:46.672106 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.666188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:54753 (local address 127.26.189.254:38853)
0807 22:42:46.666309 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.666313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.666424 (+ 111us) server_negotiation.cc:407] Connection header received
0807 22:42:46.666499 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.666502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.666544 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.666622 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.667057 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.667526 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.668303 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.668504 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.668585 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.669475 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.669478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.669480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.669500 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.670716 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.671080 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.671083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.671084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.671158 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.671400 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.671403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.671405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.671554 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.671638 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.671873 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.671968 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:42:46.676476 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:46.700971 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.693281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.26.189.254:38853)
0807 22:42:46.693397 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.693400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.693409 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:42:46.693440 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.693442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.693474 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.693536 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.694195 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.694785 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.695680 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.695930 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.696011 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.698173 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.698177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.698180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.698205 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.699463 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.699873 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.699877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.699878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.699972 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.700216 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.700220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.700221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.700384 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.700498 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.700722 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.700826 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:42:46.703552 29998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:46.706568 29998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:46.720477 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.714529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36875 (local address 127.0.0.1:47012)
0807 22:42:46.715249 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.715287 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.715418 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.716442 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.716444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.716466 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.716655 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.716665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.718031 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.718035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.719210 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.719219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.719961 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.719965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.720069 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.720107 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.720212 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.720326 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":64,"spinlock_wait_cycles":384,"thread_start_us":89,"threads_started":1}
I20260807 22:42:46.720688 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.715542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.26.189.193:43939)
0807 22:42:46.715928 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.715943 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.715967 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:42:46.716043 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.716048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.716243 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.716398 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.716728 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.717917 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.719357 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.720079 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.720244 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.720346 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.720445 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.720535 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":102,"threads_started":1}
I20260807 22:42:46.720773 30845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.714654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:34771 (local address 127.0.0.1:60948)
0807 22:42:46.715637 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.715650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.715730 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.716548 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.716551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.716568 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.716712 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.716718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.717734 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.717738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.719113 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.719125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.720413 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.720416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.720503 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.720524 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.720590 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.720654 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":40,"thread_start_us":96,"threads_started":1}
I20260807 22:42:46.720924 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.714666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.26.189.195:36875)
0807 22:42:46.716062 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.716074 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.716094 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:42:46.716164 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.716168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.716305 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.716648 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.716785 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.717923 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.719315 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.719849 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.719981 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.720134 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.720691 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.720792 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290,"thread_start_us":90,"threads_started":1}
I20260807 22:42:46.721212 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.715342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.0.0.1:42078)
0807 22:42:46.715716 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.715729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.715816 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.716405 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.716411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.716434 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.716582 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.716590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.718171 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.718175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.719221 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.719233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.720235 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.720239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.720318 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.720338 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.720414 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.720466 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":47,"spinlock_wait_cycles":4224,"thread_start_us":77,"threads_started":1}
I20260807 22:42:46.721271 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.715248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.26.189.194:34771)
0807 22:42:46.715952 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.715962 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.715980 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:42:46.716042 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.716048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.716244 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.716398 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.716805 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.717632 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.719261 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.720068 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.720220 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.720346 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.721034 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.721138 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":132,"threads_started":1}
I20260807 22:42:46.723855 30224 tablet_service.cc:1467] Processing CreateTablet for tablet b8206c1cee03413cbe56a1ec097e3a4e (DEFAULT_TABLE table=test-table [id=83a2ab03af8e4564963438576e587128]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:46.724023 30678 tablet_service.cc:1467] Processing CreateTablet for tablet b8206c1cee03413cbe56a1ec097e3a4e (DEFAULT_TABLE table=test-table [id=83a2ab03af8e4564963438576e587128]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:46.725008 30224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8206c1cee03413cbe56a1ec097e3a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.724936 30457 tablet_service.cc:1467] Processing CreateTablet for tablet b8206c1cee03413cbe56a1ec097e3a4e (DEFAULT_TABLE table=test-table [id=83a2ab03af8e4564963438576e587128]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:46.725565 30678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8206c1cee03413cbe56a1ec097e3a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.726008 30457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8206c1cee03413cbe56a1ec097e3a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.731617 30848 tablet_bootstrap.cc:492] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Bootstrap starting.
I20260807 22:42:46.733068 30853 tablet_bootstrap.cc:492] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Bootstrap starting.
I20260807 22:42:46.733474 30851 tablet_bootstrap.cc:492] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Bootstrap starting.
I20260807 22:42:46.734606 30848 tablet_bootstrap.cc:654] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:46.735509 30848 log.cc:824] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:46.735584 30851 tablet_bootstrap.cc:654] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:46.736194 30853 tablet_bootstrap.cc:654] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:46.736356 30851 log.cc:824] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:46.736966 30853 log.cc:824] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:46.737676 30848 tablet_bootstrap.cc:492] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: No bootstrap required, opened a new log
I20260807 22:42:46.737874 30848 ts_tablet_manager.cc:1397] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:42:46.738382 30851 tablet_bootstrap.cc:492] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: No bootstrap required, opened a new log
I20260807 22:42:46.738561 30851 ts_tablet_manager.cc:1397] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:42:46.739753 30853 tablet_bootstrap.cc:492] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: No bootstrap required, opened a new log
I20260807 22:42:46.739920 30853 ts_tablet_manager.cc:1397] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 22:42:46.741712 30848 raft_consensus.cc:348] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.741981 30848 raft_consensus.cc:374] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:46.742069 30848 raft_consensus.cc:729] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88df149af36419094c2429ce10f30f6, State: Initialized, Role: FOLLOWER
I20260807 22:42:46.742213 30851 raft_consensus.cc:348] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.742473 30851 raft_consensus.cc:374] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:46.742558 30851 raft_consensus.cc:729] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a1f275c6b44657ae66f48ca7021be1, State: Initialized, Role: FOLLOWER
I20260807 22:42:46.742545 30848 consensus_queue.cc:260] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.742738 30853 raft_consensus.cc:348] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.742949 30853 raft_consensus.cc:374] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:46.743006 30853 raft_consensus.cc:729] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a13898824148e98fb6c55d6c487a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:46.742980 30851 consensus_queue.cc:260] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.743395 30853 consensus_queue.cc:260] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.743522 30848 ts_tablet_manager.cc:1428] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:42:46.743629 30805 heartbeater.cc:503] Master 127.26.189.254:38853 was elected leader, sending a full tablet report...
I20260807 22:42:46.743826 30333 heartbeater.cc:503] Master 127.26.189.254:38853 was elected leader, sending a full tablet report...
I20260807 22:42:46.743888 30851 ts_tablet_manager.cc:1428] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:42:46.744251 30853 ts_tablet_manager.cc:1428] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 22:42:46.744563 30571 heartbeater.cc:503] Master 127.26.189.254:38853 was elected leader, sending a full tablet report...
I20260807 22:42:46.768149 30863 raft_consensus.cc:482] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:46.768309 30863 raft_consensus.cc:504] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.769356 30863 leader_election.cc:288] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875), a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939)
I20260807 22:42:46.773270 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.769569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36875 (local address 127.26.189.194:51657)
0807 22:42:46.769703 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.769719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.769803 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.770156 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.770160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.770183 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.770337 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.770345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.771249 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.771252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.772129 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.772138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.772843 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.772847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.772964 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.772990 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.773063 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.773144 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 22:42:46.773276 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.769829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:51657 (local address 127.26.189.195:36875)
0807 22:42:46.769958 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.769962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.769975 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:42:46.770023 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.770026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.770073 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.770142 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.770450 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.771154 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.772249 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.772744 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.772833 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.772872 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.773085 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.773152 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:42:46.773499 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.769587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.26.189.194:49265)
0807 22:42:46.770000 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.770012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.770110 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.770368 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.770371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.770386 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.770814 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.770824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.771607 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.771610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.772420 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.772431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.773112 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.773116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.773211 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.773234 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.773309 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.773370 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":134,"threads_started":1}
I20260807 22:42:46.773530 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.769693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:49265 (local address 127.26.189.193:43939)
0807 22:42:46.769852 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.769857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.770079 (+ 222us) server_negotiation.cc:407] Connection header received
0807 22:42:46.770215 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.770218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.770261 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.770359 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.770923 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.771501 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.772544 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.772984 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.773114 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.773157 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.773340 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.773414 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 22:42:46.774060 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8206c1cee03413cbe56a1ec097e3a4e" candidate_uuid: "d9a13898824148e98fb6c55d6c487a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1" is_pre_election: true
I20260807 22:42:46.774068 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8206c1cee03413cbe56a1ec097e3a4e" candidate_uuid: "d9a13898824148e98fb6c55d6c487a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88df149af36419094c2429ce10f30f6" is_pre_election: true
I20260807 22:42:46.774386 30261 raft_consensus.cc:2454] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a13898824148e98fb6c55d6c487a64 in term 0.
I20260807 22:42:46.774386 30723 raft_consensus.cc:2454] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a13898824148e98fb6c55d6c487a64 in term 0.
I20260807 22:42:46.774844 30372 leader_election.cc:302] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88df149af36419094c2429ce10f30f6, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:46.775058 30863 raft_consensus.cc:2781] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:46.775143 30863 raft_consensus.cc:482] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:46.775194 30863 raft_consensus.cc:3037] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:46.776189 30863 raft_consensus.cc:504] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.776495 30863 leader_election.cc:288] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [CANDIDATE]: Term 1 election: Requested vote from peers 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875), a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939)
I20260807 22:42:46.776804 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8206c1cee03413cbe56a1ec097e3a4e" candidate_uuid: "d9a13898824148e98fb6c55d6c487a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1"
I20260807 22:42:46.776865 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8206c1cee03413cbe56a1ec097e3a4e" candidate_uuid: "d9a13898824148e98fb6c55d6c487a64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88df149af36419094c2429ce10f30f6"
I20260807 22:42:46.776912 30723 raft_consensus.cc:3037] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:46.776970 30261 raft_consensus.cc:3037] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:46.778035 30261 raft_consensus.cc:2454] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a13898824148e98fb6c55d6c487a64 in term 1.
I20260807 22:42:46.778053 30723 raft_consensus.cc:2454] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a13898824148e98fb6c55d6c487a64 in term 1.
I20260807 22:42:46.778321 30372 leader_election.cc:302] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88df149af36419094c2429ce10f30f6, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:46.778476 30863 raft_consensus.cc:2781] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:46.778666 30863 raft_consensus.cc:686] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 1 LEADER]: Becoming Leader. State: Replica: d9a13898824148e98fb6c55d6c487a64, State: Running, Role: LEADER
I20260807 22:42:46.778956 30863 consensus_queue.cc:237] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } }
I20260807 22:42:46.781970 29998 catalog_manager.cc:5495] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9a13898824148e98fb6c55d6c487a64 (127.26.189.194). New cstate: current_term: 1 leader_uuid: "d9a13898824148e98fb6c55d6c487a64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:42:46 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142565, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:46.819129 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.811749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.26.189.254:38853)
0807 22:42:46.811883 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.811888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.812039 (+ 151us) server_negotiation.cc:407] Connection header received
0807 22:42:46.812161 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.812164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.812203 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.812265 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:46.812880 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.813383 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.814120 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.814381 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.814468 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.816542 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:46.816545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:46.816547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:46.816569 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:46.817621 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.818077 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.818079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.818080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.818147 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:46.818389 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:46.818392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:46.818393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:46.818547 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:46.818643 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.818872 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.818999 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":50}
I20260807 22:42:46.819130 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.811693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38853 (local address 127.0.0.1:37952)
0807 22:42:46.811965 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.811976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.812076 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.812300 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.812303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.812679 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:46.812770 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.812778 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.813479 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.813482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.814019 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.814026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.814474 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.814477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.814557 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.815027 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:46.815038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:46.816415 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:46.817740 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.817750 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.817755 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.817995 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.818207 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:46.818209 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:46.818211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:46.818285 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:46.818629 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:46.818634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:46.818741 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.818831 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.818948 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
I20260807 22:42:46.822533 29998 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:37952:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:42:46.823208 29998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:46.827745 30224 tablet_service.cc:1467] Processing CreateTablet for tablet bbeec526fc62420882124021e7994eb4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49bc65080f2e4cee862b6485b7149d33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:46.827872 30457 tablet_service.cc:1467] Processing CreateTablet for tablet bbeec526fc62420882124021e7994eb4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49bc65080f2e4cee862b6485b7149d33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:46.828014 30678 tablet_service.cc:1467] Processing CreateTablet for tablet bbeec526fc62420882124021e7994eb4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=49bc65080f2e4cee862b6485b7149d33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:42:46.828501 30224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbeec526fc62420882124021e7994eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.828521 30678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbeec526fc62420882124021e7994eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.828538 30457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbeec526fc62420882124021e7994eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.831091 30853 tablet_bootstrap.cc:492] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: Bootstrap starting.
I20260807 22:42:46.831419 30851 tablet_bootstrap.cc:492] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: Bootstrap starting.
I20260807 22:42:46.831682 30848 tablet_bootstrap.cc:492] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: Bootstrap starting.
I20260807 22:42:46.832170 30853 tablet_bootstrap.cc:654] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:46.832335 30851 tablet_bootstrap.cc:654] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:46.832479 30848 tablet_bootstrap.cc:654] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:46.833545 30851 tablet_bootstrap.cc:492] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: No bootstrap required, opened a new log
I20260807 22:42:46.833587 30853 tablet_bootstrap.cc:492] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: No bootstrap required, opened a new log
I20260807 22:42:46.833616 30851 ts_tablet_manager.cc:1397] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:46.833662 30853 ts_tablet_manager.cc:1397] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:42:46.833981 30851 raft_consensus.cc:348] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.834074 30851 raft_consensus.cc:374] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:46.834095 30851 raft_consensus.cc:729] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a1f275c6b44657ae66f48ca7021be1, State: Initialized, Role: FOLLOWER
I20260807 22:42:46.834070 30853 raft_consensus.cc:348] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.834192 30853 raft_consensus.cc:374] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:46.834223 30853 raft_consensus.cc:729] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a13898824148e98fb6c55d6c487a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:46.834213 30851 consensus_queue.cc:260] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.834355 30853 consensus_queue.cc:260] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.834467 30862 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.834617 30853 ts_tablet_manager.cc:1428] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:42:46.834630 30862 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.834820 30863 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.834940 30863 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.835165 30848 tablet_bootstrap.cc:492] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: No bootstrap required, opened a new log
I20260807 22:42:46.835240 30848 ts_tablet_manager.cc:1397] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:42:46.835690 30848 raft_consensus.cc:348] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.835794 30848 raft_consensus.cc:374] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:46.835822 30848 raft_consensus.cc:729] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88df149af36419094c2429ce10f30f6, State: Initialized, Role: FOLLOWER
I20260807 22:42:46.835954 30848 consensus_queue.cc:260] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.836193 30848 ts_tablet_manager.cc:1428] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:42:46.834467 30851 ts_tablet_manager.cc:1428] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:46.836421 30861 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.836539 30861 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.856293 30861 raft_consensus.cc:482] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:46.856400 30861 raft_consensus.cc:504] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.857489 30861 leader_election.cc:288] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771), 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:46.861173 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.857817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:44095 (local address 127.26.189.195:36875)
0807 22:42:46.857936 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.857940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.858216 (+ 276us) server_negotiation.cc:407] Connection header received
0807 22:42:46.858352 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.858356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.858401 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.858461 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.858738 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.859215 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.860244 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.860633 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.860738 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.860791 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.860983 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.861053 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:42:46.861243 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.857685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36875 (local address 127.26.189.193:44095)
0807 22:42:46.858133 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.858147 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.858249 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.858478 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.858480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.858496 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.858635 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.858640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.859312 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.859316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.860125 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.860134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.860768 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.860772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.860873 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.860891 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.860956 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.861020 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":120,"thread_start_us":109,"threads_started":1}
I20260807 22:42:46.861333 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.857856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:40719 (local address 127.26.189.194:34771)
0807 22:42:46.857961 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.857966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.858162 (+ 196us) server_negotiation.cc:407] Connection header received
0807 22:42:46.858276 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.858279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.858347 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.858409 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:46.858654 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.859145 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.860434 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.860840 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.860942 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.860990 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.861141 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.861216 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:42:46.861620 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.857685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:34771 (local address 127.26.189.193:40719)
0807 22:42:46.858056 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.858072 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.858175 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.858427 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.858430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.858446 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:46.858555 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.858560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.859242 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.859246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.860320 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.860330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.860943 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.860946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.861039 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.861057 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.861126 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.861239 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":36,"thread_start_us":98,"threads_started":1}
I20260807 22:42:46.861725 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeec526fc62420882124021e7994eb4" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1" is_pre_election: true
I20260807 22:42:46.861877 30723 raft_consensus.cc:2454] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 0.
I20260807 22:42:46.862215 30150 leader_election.cc:302] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a1f275c6b44657ae66f48ca7021be1, a88df149af36419094c2429ce10f30f6; no voters:
I20260807 22:42:46.862195 30491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeec526fc62420882124021e7994eb4" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64" is_pre_election: true
I20260807 22:42:46.862448 30491 raft_consensus.cc:2454] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 0.
I20260807 22:42:46.862501 30861 raft_consensus.cc:2781] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:46.862558 30861 raft_consensus.cc:482] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:46.862600 30861 raft_consensus.cc:3037] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:46.863556 30861 raft_consensus.cc:504] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.863855 30861 leader_election.cc:288] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 election: Requested vote from peers d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771), 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:46.864151 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeec526fc62420882124021e7994eb4" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1"
I20260807 22:42:46.864148 30491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeec526fc62420882124021e7994eb4" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64"
I20260807 22:42:46.864270 30491 raft_consensus.cc:3037] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:46.864269 30723 raft_consensus.cc:3037] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:46.865407 30723 raft_consensus.cc:2454] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 1.
I20260807 22:42:46.865411 30491 raft_consensus.cc:2454] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 1.
I20260807 22:42:46.865797 30149 leader_election.cc:302] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88df149af36419094c2429ce10f30f6, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:46.866091 30861 raft_consensus.cc:2781] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:46.866307 30861 raft_consensus.cc:686] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [term 1 LEADER]: Becoming Leader. State: Replica: a88df149af36419094c2429ce10f30f6, State: Running, Role: LEADER
I20260807 22:42:46.866636 30861 consensus_queue.cc:237] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:46.868683 30889 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: TxnStatusTablet state changed. Reason: New leader a88df149af36419094c2429ce10f30f6. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.868770 30889 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:46.869028 30892 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:42:46.869215 30892 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:42:46.869851 29999 catalog_manager.cc:5495] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a88df149af36419094c2429ce10f30f6 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } health_report { overall_health: UNKNOWN } } }
I20260807 22:42:46.885300 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.881694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.0.0.1:42088)
0807 22:42:46.881929 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.881942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.882050 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.882360 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.882363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.882372 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:46.882471 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.882477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.883156 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.883159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.883782 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.883790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.884221 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.884225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.884328 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.884376 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.884998 (+ 622us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.885015 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.885071 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.885172 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260807 22:42:46.885315 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.881764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.26.189.193:43939)
0807 22:42:46.881876 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.881881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.881994 (+ 113us) server_negotiation.cc:407] Connection header received
0807 22:42:46.882179 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.882182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.882232 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.882336 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:46.882566 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.883052 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.883892 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.884111 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.884207 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.884506 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.884907 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.885006 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.885093 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.885149 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:42:46.890029 30491 raft_consensus.cc:1261] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Refusing update from remote peer a88df149af36419094c2429ce10f30f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:46.890033 30723 raft_consensus.cc:1261] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Refusing update from remote peer a88df149af36419094c2429ce10f30f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:46.890652 30889 consensus_queue.cc:1035] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:46.890822 30861 consensus_queue.cc:1035] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:46.893781 30863 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: TxnStatusTablet state changed. Reason: New leader a88df149af36419094c2429ce10f30f6. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.893893 30863 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.894575 30862 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: TxnStatusTablet state changed. Reason: New leader a88df149af36419094c2429ce10f30f6. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.894703 30862 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.897341 30861 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.897447 30861 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:46.898579 30863 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.898696 30863 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P d9a13898824148e98fb6c55d6c487a64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.900420 30906 mvcc.cc:205] Tried to move back new op lower bound from 7316039953975033856 to 7316039953888014336. Current Snapshot: MvccSnapshot[applied={T|T < 7316039953975033856}]
I20260807 22:42:46.904296 30889 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.904422 30889 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P a88df149af36419094c2429ce10f30f6: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:42:46.905172 30905 mvcc.cc:205] Tried to move back new op lower bound from 7316039953975033856 to 7316039953888014336. Current Snapshot: MvccSnapshot[applied={T|T < 7316039953975033856}]
I20260807 22:42:46.905380 30862 tablet_replica.cc:439] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } } }
I20260807 22:42:46.905480 30862 tablet_replica.cc:442] T bbeec526fc62420882124021e7994eb4 P 90a1f275c6b44657ae66f48ca7021be1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:42:46.907372 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.902842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.26.189.194:34771)
0807 22:42:46.902944 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.902949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.902961 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:42:46.903024 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.903027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.903074 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.903156 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:46.903402 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.904044 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.905175 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.905490 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.905589 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.906255 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.906675 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.906775 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.907202 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.907250 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 22:42:46.911765 30806 tablet.cc:2369] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:46.919706 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.916532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53251 (local address 127.26.189.193:43939)
0807 22:42:46.916647 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.916651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.916685 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:42:46.916814 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.916816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.916871 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.916928 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:46.917164 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.917644 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.918798 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.919001 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.919072 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.919374 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.919433 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.919475 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.919567 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.919610 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:42:46.919768 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.916450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.26.189.194:53251)
0807 22:42:46.916620 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.916636 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.916730 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.916947 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.916950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.916957 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:46.917071 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.917076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.917728 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.917731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.918696 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.918707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.919096 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.919100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.919192 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.919241 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.919495 (+ 254us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.919501 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.919561 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.919638 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 22:42:46.926802 30261 raft_consensus.cc:1261] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Refusing update from remote peer d9a13898824148e98fb6c55d6c487a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:46.927248 30723 raft_consensus.cc:1261] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Refusing update from remote peer d9a13898824148e98fb6c55d6c487a64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:46.927537 30863 consensus_queue.cc:1035] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:46.927727 30865 consensus_queue.cc:1035] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:46.932296 30901 mvcc.cc:205] Tried to move back new op lower bound from 7316039954127560704 to 7316039953528897536. Current Snapshot: MvccSnapshot[applied={T|T < 7316039954127560704}]
I20260807 22:42:46.950544 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.947169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:55295 (local address 127.26.189.194:34771)
0807 22:42:46.947294 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:46.947299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:46.947517 (+ 218us) server_negotiation.cc:407] Connection header received
0807 22:42:46.947638 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:46.947640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:46.947679 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:46.947737 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:46.947970 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.948495 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.949411 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.949725 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.949826 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.950101 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.950174 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.950227 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:46.950359 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 22:42:46.950416 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:42:46.950580 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:46.947101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:34771 (local address 127.26.189.193:55295)
0807 22:42:46.947434 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:46.947448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:46.947550 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:46.947758 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:46.947761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:46.947768 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:46.947886 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.947892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.948626 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.948629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.949309 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:46.949319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:46.949825 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:46.949829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:46.949931 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:46.949983 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:46.950244 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:46.950249 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:42:46.950347 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:42:46.950422 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
I20260807 22:42:46.983209 29999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:46.983385 29999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:46.986117 29999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Sending DeleteTablet for 3 replicas of tablet b8206c1cee03413cbe56a1ec097e3a4e
I20260807 22:42:46.987082 30678 tablet_service.cc:1514] Processing DeleteTablet for tablet b8206c1cee03413cbe56a1ec097e3a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:46 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260807 22:42:46.987210 30457 tablet_service.cc:1514] Processing DeleteTablet for tablet b8206c1cee03413cbe56a1ec097e3a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:46 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60948
I20260807 22:42:46.988098 30224 tablet_service.cc:1514] Processing DeleteTablet for tablet b8206c1cee03413cbe56a1ec097e3a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:46 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:42078
I20260807 22:42:46.988803 30933 tablet_replica.cc:330] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: stopping tablet replica
I20260807 22:42:46.989059 30934 tablet_replica.cc:330] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: stopping tablet replica
I20260807 22:42:46.989410 30933 raft_consensus.cc:2229] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:42:46.989336 29999 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:46.989732 29999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:46.989892 30934 raft_consensus.cc:2229] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:46.990120 30934 raft_consensus.cc:2258] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:46.990625 30933 raft_consensus.cc:2258] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:46.992281 30934 ts_tablet_manager.cc:1905] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:46.993234 30933 ts_tablet_manager.cc:1905] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:46.995188 30457 tablet_service.cc:1467] Processing CreateTablet for tablet 1e536ddd11d34da3a4f87cf4592308a2 (DEFAULT_TABLE table=test-table [id=9f1456db3b4f49ed82dfcddded9db599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:46.995451 30457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e536ddd11d34da3a4f87cf4592308a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.997838 30678 tablet_service.cc:1467] Processing CreateTablet for tablet 1e536ddd11d34da3a4f87cf4592308a2 (DEFAULT_TABLE table=test-table [id=9f1456db3b4f49ed82dfcddded9db599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:46.998095 30678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e536ddd11d34da3a4f87cf4592308a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:46.998133 30933 ts_tablet_manager.cc:1918] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:46.998208 30933 log.cc:1196] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/b8206c1cee03413cbe56a1ec097e3a4e
I20260807 22:42:46.998504 30853 tablet_bootstrap.cc:492] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Bootstrap starting.
I20260807 22:42:46.998528 30933 ts_tablet_manager.cc:1939] T b8206c1cee03413cbe56a1ec097e3a4e P d9a13898824148e98fb6c55d6c487a64: Deleting consensus metadata
I20260807 22:42:46.999395 30853 tablet_bootstrap.cc:654] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:47.000528 30934 ts_tablet_manager.cc:1918] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:47.000661 30934 log.cc:1196] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/b8206c1cee03413cbe56a1ec097e3a4e
I20260807 22:42:47.000859 30224 tablet_service.cc:1467] Processing CreateTablet for tablet 1e536ddd11d34da3a4f87cf4592308a2 (DEFAULT_TABLE table=test-table [id=9f1456db3b4f49ed82dfcddded9db599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:47.001006 30934 ts_tablet_manager.cc:1939] T b8206c1cee03413cbe56a1ec097e3a4e P 90a1f275c6b44657ae66f48ca7021be1: Deleting consensus metadata
I20260807 22:42:47.001161 30224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e536ddd11d34da3a4f87cf4592308a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:47.003616 30848 tablet_bootstrap.cc:492] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Bootstrap starting.
I20260807 22:42:47.004372 30851 tablet_bootstrap.cc:492] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Bootstrap starting.
I20260807 22:42:47.004529 30848 tablet_bootstrap.cc:654] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:47.004820 29976 catalog_manager.cc:4841] TS 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875): tablet b8206c1cee03413cbe56a1ec097e3a4e (table test-table [id=83a2ab03af8e4564963438576e587128]) successfully deleted
I20260807 22:42:47.005089 29974 catalog_manager.cc:4841] TS d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771): tablet b8206c1cee03413cbe56a1ec097e3a4e (table test-table [id=83a2ab03af8e4564963438576e587128]) successfully deleted
I20260807 22:42:47.005278 30851 tablet_bootstrap.cc:654] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:47.005863 30935 tablet_replica.cc:330] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: stopping tablet replica
I20260807 22:42:47.006186 30935 raft_consensus.cc:2229] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:47.006449 30935 raft_consensus.cc:2258] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:47.007123 30853 tablet_bootstrap.cc:492] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: No bootstrap required, opened a new log
I20260807 22:42:47.007220 30853 ts_tablet_manager.cc:1397] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260807 22:42:47.007357 30851 tablet_bootstrap.cc:492] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: No bootstrap required, opened a new log
I20260807 22:42:47.007434 30851 ts_tablet_manager.cc:1397] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:42:47.007741 30853 raft_consensus.cc:348] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.007865 30853 raft_consensus.cc:374] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:47.007903 30853 raft_consensus.cc:729] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a13898824148e98fb6c55d6c487a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:47.008033 30853 consensus_queue.cc:260] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.008117 30851 raft_consensus.cc:348] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.008240 30851 raft_consensus.cc:374] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:47.008275 30851 raft_consensus.cc:729] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a1f275c6b44657ae66f48ca7021be1, State: Initialized, Role: FOLLOWER
I20260807 22:42:47.008335 30853 ts_tablet_manager.cc:1428] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:42:47.008388 30851 consensus_queue.cc:260] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.009148 30851 ts_tablet_manager.cc:1428] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:47.009977 30848 tablet_bootstrap.cc:492] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: No bootstrap required, opened a new log
I20260807 22:42:47.010066 30848 ts_tablet_manager.cc:1397] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 22:42:47.010533 30848 raft_consensus.cc:348] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.010638 30848 raft_consensus.cc:374] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:47.010669 30848 raft_consensus.cc:729] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88df149af36419094c2429ce10f30f6, State: Initialized, Role: FOLLOWER
I20260807 22:42:47.010780 30848 consensus_queue.cc:260] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.011032 30848 ts_tablet_manager.cc:1428] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:47.013317 30935 ts_tablet_manager.cc:1905] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:47.016680 30935 ts_tablet_manager.cc:1918] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:47.016844 30935 log.cc:1196] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/b8206c1cee03413cbe56a1ec097e3a4e
I20260807 22:42:47.017261 30935 ts_tablet_manager.cc:1939] T b8206c1cee03413cbe56a1ec097e3a4e P a88df149af36419094c2429ce10f30f6: Deleting consensus metadata
I20260807 22:42:47.018262 29973 catalog_manager.cc:4841] TS a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939): tablet b8206c1cee03413cbe56a1ec097e3a4e (table test-table [id=83a2ab03af8e4564963438576e587128]) successfully deleted
I20260807 22:42:47.058077 30862 raft_consensus.cc:482] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:47.058233 30862 raft_consensus.cc:504] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.059495 30862 leader_election.cc:288] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939), d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771)
I20260807 22:42:47.064576 30943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.059686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:34771 (local address 127.26.189.195:60093)
0807 22:42:47.060150 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:47.060163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:47.060282 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:47.060604 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:47.060607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:47.060626 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:47.060752 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.060759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.061759 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.061763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.063067 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.063079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.064070 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.064073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.064185 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.064246 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 22:42:47.064321 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:42:47.064398 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":126,"thread_start_us":83,"threads_started":1}
I20260807 22:42:47.064651 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.059840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:60093 (local address 127.26.189.194:34771)
0807 22:42:47.059958 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:47.059962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:47.060263 (+ 301us) server_negotiation.cc:407] Connection header received
0807 22:42:47.060366 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:47.060369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:47.060431 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:47.060496 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:47.060878 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.061662 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.063218 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.063777 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.063871 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.063929 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:47.064346 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 22:42:47.064427 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:42:47.066624 30491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e536ddd11d34da3a4f87cf4592308a2" candidate_uuid: "90a1f275c6b44657ae66f48ca7021be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64" is_pre_election: true
I20260807 22:42:47.066805 30491 raft_consensus.cc:2454] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a1f275c6b44657ae66f48ca7021be1 in term 0.
I20260807 22:42:47.067116 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.060094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:55053 (local address 127.26.189.193:43939)
0807 22:42:47.060208 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:47.060212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:47.060224 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:42:47.060279 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:47.060282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:47.060328 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:47.060401 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:42:47.060666 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.061241 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.062718 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.063861 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.066415 (+ 2554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.066469 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:47.066863 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 22:42:47.066980 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:42:47.067727 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.059686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.26.189.195:55053)
0807 22:42:47.059790 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:47.059805 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:47.059892 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:47.060410 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:47.060413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:47.060427 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:42:47.060558 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.060564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.061341 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.061344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.062566 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.062577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.066440 (+ 3863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.066444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.066618 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.066671 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:42:47.066806 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 22:42:47.067609 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 22:42:47.068172 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e536ddd11d34da3a4f87cf4592308a2" candidate_uuid: "90a1f275c6b44657ae66f48ca7021be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88df149af36419094c2429ce10f30f6" is_pre_election: true
I20260807 22:42:47.068316 30261 raft_consensus.cc:2454] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a1f275c6b44657ae66f48ca7021be1 in term 0.
I20260807 22:42:47.068706 30612 leader_election.cc:302] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a1f275c6b44657ae66f48ca7021be1, a88df149af36419094c2429ce10f30f6; no voters:
I20260807 22:42:47.068988 30862 raft_consensus.cc:2781] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:47.069046 30862 raft_consensus.cc:482] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:47.069088 30862 raft_consensus.cc:3037] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:47.070397 30862 raft_consensus.cc:504] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.070775 30862 leader_election.cc:288] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [CANDIDATE]: Term 1 election: Requested vote from peers a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939), d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771)
I20260807 22:42:47.071983 30491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e536ddd11d34da3a4f87cf4592308a2" candidate_uuid: "90a1f275c6b44657ae66f48ca7021be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64"
I20260807 22:42:47.071992 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e536ddd11d34da3a4f87cf4592308a2" candidate_uuid: "90a1f275c6b44657ae66f48ca7021be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88df149af36419094c2429ce10f30f6"
I20260807 22:42:47.072110 30261 raft_consensus.cc:3037] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:47.072126 30491 raft_consensus.cc:3037] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:47.073263 30491 raft_consensus.cc:2454] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a1f275c6b44657ae66f48ca7021be1 in term 1.
I20260807 22:42:47.073555 30261 raft_consensus.cc:2454] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a1f275c6b44657ae66f48ca7021be1 in term 1.
I20260807 22:42:47.073621 30613 leader_election.cc:302] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a1f275c6b44657ae66f48ca7021be1, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:47.074025 30862 raft_consensus.cc:2781] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:47.074255 30862 raft_consensus.cc:686] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 LEADER]: Becoming Leader. State: Replica: 90a1f275c6b44657ae66f48ca7021be1, State: Running, Role: LEADER
I20260807 22:42:47.075140 30862 consensus_queue.cc:237] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:47.077481 29999 catalog_manager.cc:5495] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195). New cstate: current_term: 1 leader_uuid: "90a1f275c6b44657ae66f48ca7021be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } health_report { overall_health: HEALTHY } } }
I20260807 22:42:47.100188 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.096421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.26.189.195:36875)
0807 22:42:47.096525 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:47.096529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:47.096734 (+ 205us) server_negotiation.cc:407] Connection header received
0807 22:42:47.096798 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:47.096800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:47.096852 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:47.096925 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:47.097196 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.097676 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.098856 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.099064 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.099143 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.099398 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:47.099795 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:47.099877 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:47.099998 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 22:42:47.100046 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:42:47.110257 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.106867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.26.189.195:55741)
0807 22:42:47.107007 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:47.107024 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:47.107111 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:47.107335 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:47.107337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:47.107342 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:47.107442 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.107446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.108406 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.108408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.109108 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.109119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.109649 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.109651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.109713 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.109754 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:47.110033 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:47.110036 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:42:47.110076 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 22:42:47.110148 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:42:47.110325 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.106947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:55741 (local address 127.26.189.193:43939)
0807 22:42:47.107094 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:47.107099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:47.107116 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:42:47.107198 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:47.107201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:47.107256 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:47.107336 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:47.107552 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.108303 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.109216 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.109549 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.109663 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.109869 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:47.109957 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:47.110037 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:47.110093 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:42:47.110156 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:42:47.116302 30261 raft_consensus.cc:1261] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Refusing update from remote peer 90a1f275c6b44657ae66f48ca7021be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:47.116324 30491 raft_consensus.cc:1261] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Refusing update from remote peer 90a1f275c6b44657ae66f48ca7021be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:47.116814 30944 consensus_queue.cc:1035] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:47.116981 30862 consensus_queue.cc:1035] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:47.135246 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.131694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:48111 (local address 127.26.189.195:36875)
0807 22:42:47.131796 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:47.131800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:47.131812 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:42:47.131849 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:47.131852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:47.131898 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:47.131974 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:47.132237 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.132872 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.134018 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.134385 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.134464 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.134773 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:47.134856 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:47.134924 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:47.135074 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 22:42:47.135129 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:42:47.135262 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:47.131182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36875 (local address 127.26.189.193:48111)
0807 22:42:47.131296 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:47.131311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:47.131389 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:47.132014 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:47.132018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:47.132027 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:47.132133 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.132140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.132993 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.132997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.133904 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:47.133917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:47.134500 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:47.134504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:47.134591 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:47.134625 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:47.134967 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:47.134975 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:42:47.135043 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 22:42:47.135131 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 22:42:47.136039 30905 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:42:48.691957 29998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:48.692106 29998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:48.694284 29998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Sending DeleteTablet for 3 replicas of tablet 1e536ddd11d34da3a4f87cf4592308a2
I20260807 22:42:48.694823 30224 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e536ddd11d34da3a4f87cf4592308a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:48 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:42078
I20260807 22:42:48.694859 30678 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e536ddd11d34da3a4f87cf4592308a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:48 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260807 22:42:48.694936 30457 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e536ddd11d34da3a4f87cf4592308a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:48 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60948
I20260807 22:42:48.695228 31004 tablet_replica.cc:330] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: stopping tablet replica
I20260807 22:42:48.695355 31004 raft_consensus.cc:2229] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:48.697903 31004 raft_consensus.cc:2258] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:48.698169 31005 tablet_replica.cc:330] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: stopping tablet replica
I20260807 22:42:48.698326 31005 raft_consensus.cc:2229] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:42:48.698453 31008 tablet_replica.cc:330] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: stopping tablet replica
I20260807 22:42:48.698580 31004 ts_tablet_manager.cc:1905] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:48.698580 31008 raft_consensus.cc:2229] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:48.698629 31005 raft_consensus.cc:2258] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:48.698704 31008 raft_consensus.cc:2258] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:48.699268 31005 ts_tablet_manager.cc:1905] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:48.701252 31005 ts_tablet_manager.cc:1918] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:42:48.701275 31004 ts_tablet_manager.cc:1918] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:42:48.701323 31005 log.cc:1196] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/1e536ddd11d34da3a4f87cf4592308a2
I20260807 22:42:48.701337 31004 log.cc:1196] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/1e536ddd11d34da3a4f87cf4592308a2
I20260807 22:42:48.701562 31004 ts_tablet_manager.cc:1939] T 1e536ddd11d34da3a4f87cf4592308a2 P a88df149af36419094c2429ce10f30f6: Deleting consensus metadata
I20260807 22:42:48.701761 31008 ts_tablet_manager.cc:1905] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:48.702306 29973 catalog_manager.cc:4841] TS a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939): tablet 1e536ddd11d34da3a4f87cf4592308a2 (table test-table [id=9f1456db3b4f49ed82dfcddded9db599]) successfully deleted
I20260807 22:42:48.704173 31008 ts_tablet_manager.cc:1918] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:42:48.704254 31008 log.cc:1196] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/1e536ddd11d34da3a4f87cf4592308a2
I20260807 22:42:48.704491 31008 ts_tablet_manager.cc:1939] T 1e536ddd11d34da3a4f87cf4592308a2 P d9a13898824148e98fb6c55d6c487a64: Deleting consensus metadata
I20260807 22:42:48.705150 29974 catalog_manager.cc:4841] TS d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771): tablet 1e536ddd11d34da3a4f87cf4592308a2 (table test-table [id=9f1456db3b4f49ed82dfcddded9db599]) successfully deleted
I20260807 22:42:48.705286 31005 ts_tablet_manager.cc:1939] T 1e536ddd11d34da3a4f87cf4592308a2 P 90a1f275c6b44657ae66f48ca7021be1: Deleting consensus metadata
I20260807 22:42:48.705897 29976 catalog_manager.cc:4841] TS 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875): tablet 1e536ddd11d34da3a4f87cf4592308a2 (table test-table [id=9f1456db3b4f49ed82dfcddded9db599]) successfully deleted
Aug 07 22:42:48 dist-test-slave-4tdj krb5kdc[29931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:42:48 dist-test-slave-4tdj krb5kdc[29931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:48.727739 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:48.715192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.26.189.254:38853)
0807 22:42:48.715452 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:48.715461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:48.715474 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:42:48.715613 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:48.715616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:48.715665 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:48.715740 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:48.716397 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:48.716941 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:48.717735 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:48.717966 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:48.718051 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:48.725104 (+ 7053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:48.725107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:48.725109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:48.725134 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:48.726448 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:48.726803 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:48.726806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:48.726807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:48.726882 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:48.727064 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:48.727066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:48.727067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:48.727235 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:48.727341 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:48.727499 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 22:42:48.727608 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260807 22:42:48.730497 29998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:48.730988 29998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:48.735690 30224 tablet_service.cc:1467] Processing CreateTablet for tablet 624ec0f152f54760826741de07c0cd7e (DEFAULT_TABLE table=test-table [id=88ed5e4b30e74440892f4697664adf89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:48.736006 30224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 624ec0f152f54760826741de07c0cd7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:48.736017 30678 tablet_service.cc:1467] Processing CreateTablet for tablet 624ec0f152f54760826741de07c0cd7e (DEFAULT_TABLE table=test-table [id=88ed5e4b30e74440892f4697664adf89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:48.736266 30678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 624ec0f152f54760826741de07c0cd7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:48.738674 31024 tablet_bootstrap.cc:492] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Bootstrap starting.
I20260807 22:42:48.738862 30457 tablet_service.cc:1467] Processing CreateTablet for tablet 624ec0f152f54760826741de07c0cd7e (DEFAULT_TABLE table=test-table [id=88ed5e4b30e74440892f4697664adf89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:48.739089 30457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 624ec0f152f54760826741de07c0cd7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:48.739466 31024 tablet_bootstrap.cc:654] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:48.739974 31023 tablet_bootstrap.cc:492] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Bootstrap starting.
I20260807 22:42:48.740617 31024 tablet_bootstrap.cc:492] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: No bootstrap required, opened a new log
I20260807 22:42:48.740697 31024 ts_tablet_manager.cc:1397] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:48.741125 31028 tablet_bootstrap.cc:492] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Bootstrap starting.
I20260807 22:42:48.741137 31024 raft_consensus.cc:348] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.741255 31024 raft_consensus.cc:374] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:48.741289 31024 raft_consensus.cc:729] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88df149af36419094c2429ce10f30f6, State: Initialized, Role: FOLLOWER
I20260807 22:42:48.741390 31023 tablet_bootstrap.cc:654] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:48.741441 31024 consensus_queue.cc:260] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.741817 31024 ts_tablet_manager.cc:1428] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:48.741945 31028 tablet_bootstrap.cc:654] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:48.742787 31023 tablet_bootstrap.cc:492] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: No bootstrap required, opened a new log
I20260807 22:42:48.742880 31023 ts_tablet_manager.cc:1397] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:42:48.743136 31028 tablet_bootstrap.cc:492] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: No bootstrap required, opened a new log
I20260807 22:42:48.743227 31028 ts_tablet_manager.cc:1397] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:48.743405 31023 raft_consensus.cc:348] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.743510 31023 raft_consensus.cc:374] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:48.743597 31023 raft_consensus.cc:729] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a1f275c6b44657ae66f48ca7021be1, State: Initialized, Role: FOLLOWER
I20260807 22:42:48.743665 31028 raft_consensus.cc:348] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.743774 31028 raft_consensus.cc:374] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:48.743811 31028 raft_consensus.cc:729] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a13898824148e98fb6c55d6c487a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:48.743927 31028 consensus_queue.cc:260] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.743930 31023 consensus_queue.cc:260] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.744263 31028 ts_tablet_manager.cc:1428] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:48.744870 31023 ts_tablet_manager.cc:1428] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260807 22:42:48.961195 30574 tablet.cc:2369] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:48.964197 30985 raft_consensus.cc:482] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:48.964340 30985 raft_consensus.cc:504] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.964776 30985 leader_election.cc:288] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771), 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:48.965250 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "624ec0f152f54760826741de07c0cd7e" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1" is_pre_election: true
I20260807 22:42:48.965255 30490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "624ec0f152f54760826741de07c0cd7e" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64" is_pre_election: true
I20260807 22:42:48.965404 30723 raft_consensus.cc:2454] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 0.
I20260807 22:42:48.965415 30490 raft_consensus.cc:2454] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 0.
I20260807 22:42:48.965757 30150 leader_election.cc:302] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a1f275c6b44657ae66f48ca7021be1, a88df149af36419094c2429ce10f30f6; no voters:
I20260807 22:42:48.966010 30985 raft_consensus.cc:2781] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:48.966060 30985 raft_consensus.cc:482] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:48.966085 30985 raft_consensus.cc:3037] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:48.967248 30985 raft_consensus.cc:504] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.967574 30985 leader_election.cc:288] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 election: Requested vote from peers d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771), 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:48.967965 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "624ec0f152f54760826741de07c0cd7e" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1"
I20260807 22:42:48.967972 30490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "624ec0f152f54760826741de07c0cd7e" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64"
I20260807 22:42:48.968087 30490 raft_consensus.cc:3037] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:48.968086 30723 raft_consensus.cc:3037] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:48.969188 30723 raft_consensus.cc:2454] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 1.
I20260807 22:42:48.969198 30490 raft_consensus.cc:2454] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 1.
I20260807 22:42:48.969461 30150 leader_election.cc:302] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 90a1f275c6b44657ae66f48ca7021be1, a88df149af36419094c2429ce10f30f6, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:48.969625 30985 raft_consensus.cc:2781] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:48.969723 30985 raft_consensus.cc:686] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 1 LEADER]: Becoming Leader. State: Replica: a88df149af36419094c2429ce10f30f6, State: Running, Role: LEADER
I20260807 22:42:48.969879 30985 consensus_queue.cc:237] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:48.971563 29999 catalog_manager.cc:5495] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a88df149af36419094c2429ce10f30f6 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } health_report { overall_health: UNKNOWN } } }
W20260807 22:42:48.986685 30334 tablet.cc:2369] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:42:49.037220 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.034197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.26.189.193:43939)
0807 22:42:49.034580 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:49.034607 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:49.034617 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:42:49.034653 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:49.034656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:49.034695 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:49.034768 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:49.035019 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.035549 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.036328 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.036563 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.036673 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.036857 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:49.036922 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:49.036968 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:49.037029 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 22:42:49.037071 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":139,"threads_started":1}
I20260807 22:42:49.045079 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.042649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:45795 (local address 127.26.189.193:43939)
0807 22:42:49.042747 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:49.042750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:49.042842 (+ 92us) server_negotiation.cc:407] Connection header received
0807 22:42:49.042932 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:49.042934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:49.042971 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:49.043028 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:42:49.043186 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.043648 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.044364 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.044554 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.044624 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.044783 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:42:49.044836 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:42:49.044880 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:49.044938 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 22:42:49.044973 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:42:49.045099 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:49.042513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.26.189.193:45795)
0807 22:42:49.042771 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:49.042783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:49.042867 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:49.043038 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:49.043040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:49.043046 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:42:49.043122 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:49.043126 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.043717 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.043719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:49.044286 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:49.044292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:49.044624 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:49.044626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:49.044689 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:49.044711 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:42:49.044888 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:42:49.044891 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:42:49.044931 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 22:42:49.044988 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":108,"threads_started":1}
I20260807 22:42:49.050209 30723 raft_consensus.cc:1261] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Refusing update from remote peer a88df149af36419094c2429ce10f30f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:49.050232 30490 raft_consensus.cc:1261] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Refusing update from remote peer a88df149af36419094c2429ce10f30f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:49.050701 30973 consensus_queue.cc:1035] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:49.050846 30969 consensus_queue.cc:1035] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:49.083731 29998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:49.083876 29998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:49.085696 29998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Sending DeleteTablet for 3 replicas of tablet 624ec0f152f54760826741de07c0cd7e
I20260807 22:42:49.086177 30224 tablet_service.cc:1514] Processing DeleteTablet for tablet 624ec0f152f54760826741de07c0cd7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:49 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:42078
I20260807 22:42:49.086207 30457 tablet_service.cc:1514] Processing DeleteTablet for tablet 624ec0f152f54760826741de07c0cd7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:49 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60948
I20260807 22:42:49.086370 31004 tablet_replica.cc:330] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: stopping tablet replica
I20260807 22:42:49.086383 30678 tablet_service.cc:1514] Processing DeleteTablet for tablet 624ec0f152f54760826741de07c0cd7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:49 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260807 22:42:49.086480 31004 raft_consensus.cc:2229] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:42:49.086545 31008 tablet_replica.cc:330] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: stopping tablet replica
I20260807 22:42:49.086544 31005 tablet_replica.cc:330] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: stopping tablet replica
I20260807 22:42:49.086652 31008 raft_consensus.cc:2229] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:49.086655 31005 raft_consensus.cc:2229] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:49.086748 31008 raft_consensus.cc:2258] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:49.086748 31005 raft_consensus.cc:2258] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:49.086740 31004 raft_consensus.cc:2258] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:49.087663 31004 ts_tablet_manager.cc:1905] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:49.087662 31008 ts_tablet_manager.cc:1905] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:49.087749 29998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:42:49.088109 29998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:42:49.089988 31008 ts_tablet_manager.cc:1918] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:49.090040 31008 log.cc:1196] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/624ec0f152f54760826741de07c0cd7e
I20260807 22:42:49.090221 31008 ts_tablet_manager.cc:1939] T 624ec0f152f54760826741de07c0cd7e P d9a13898824148e98fb6c55d6c487a64: Deleting consensus metadata
I20260807 22:42:49.090924 29974 catalog_manager.cc:4841] TS d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771): tablet 624ec0f152f54760826741de07c0cd7e (table test-table [id=88ed5e4b30e74440892f4697664adf89]) successfully deleted
I20260807 22:42:49.090252 31004 ts_tablet_manager.cc:1918] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:49.091463 31004 log.cc:1196] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/624ec0f152f54760826741de07c0cd7e
I20260807 22:42:49.091773 31004 ts_tablet_manager.cc:1939] T 624ec0f152f54760826741de07c0cd7e P a88df149af36419094c2429ce10f30f6: Deleting consensus metadata
I20260807 22:42:49.092420 29973 catalog_manager.cc:4841] TS a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939): tablet 624ec0f152f54760826741de07c0cd7e (table test-table [id=88ed5e4b30e74440892f4697664adf89]) successfully deleted
I20260807 22:42:49.092803 31005 ts_tablet_manager.cc:1905] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:49.095261 31005 ts_tablet_manager.cc:1918] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:42:49.095337 31005 log.cc:1196] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/624ec0f152f54760826741de07c0cd7e
I20260807 22:42:49.095535 30457 tablet_service.cc:1467] Processing CreateTablet for tablet 5356107706c3414dabfc4698cf029670 (DEFAULT_TABLE table=test-table [id=441fc8aafa9c40069c3927304f3af26c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:49.095752 30678 tablet_service.cc:1467] Processing CreateTablet for tablet 5356107706c3414dabfc4698cf029670 (DEFAULT_TABLE table=test-table [id=441fc8aafa9c40069c3927304f3af26c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:49.095805 30457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5356107706c3414dabfc4698cf029670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:49.095796 30224 tablet_service.cc:1467] Processing CreateTablet for tablet 5356107706c3414dabfc4698cf029670 (DEFAULT_TABLE table=test-table [id=441fc8aafa9c40069c3927304f3af26c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:42:49.095983 30678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5356107706c3414dabfc4698cf029670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:49.096021 30224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5356107706c3414dabfc4698cf029670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:49.097906 31023 tablet_bootstrap.cc:492] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Bootstrap starting.
I20260807 22:42:49.097941 31028 tablet_bootstrap.cc:492] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Bootstrap starting.
I20260807 22:42:49.098719 31023 tablet_bootstrap.cc:654] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:49.098791 31028 tablet_bootstrap.cc:654] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:49.099328 31005 ts_tablet_manager.cc:1939] T 624ec0f152f54760826741de07c0cd7e P 90a1f275c6b44657ae66f48ca7021be1: Deleting consensus metadata
I20260807 22:42:49.099786 31028 tablet_bootstrap.cc:492] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: No bootstrap required, opened a new log
I20260807 22:42:49.099869 31028 ts_tablet_manager.cc:1397] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:49.099927 29976 catalog_manager.cc:4841] TS 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875): tablet 624ec0f152f54760826741de07c0cd7e (table test-table [id=88ed5e4b30e74440892f4697664adf89]) successfully deleted
I20260807 22:42:49.100208 31028 raft_consensus.cc:348] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.100293 31024 tablet_bootstrap.cc:492] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6: Bootstrap starting.
I20260807 22:42:49.100310 31028 raft_consensus.cc:374] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:49.100340 31028 raft_consensus.cc:729] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a13898824148e98fb6c55d6c487a64, State: Initialized, Role: FOLLOWER
I20260807 22:42:49.100450 31028 consensus_queue.cc:260] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.100704 31028 ts_tablet_manager.cc:1428] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:49.101047 31023 tablet_bootstrap.cc:492] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: No bootstrap required, opened a new log
I20260807 22:42:49.101076 31024 tablet_bootstrap.cc:654] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:49.101130 31023 ts_tablet_manager.cc:1397] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:42:49.101483 31023 raft_consensus.cc:348] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.101578 31023 raft_consensus.cc:374] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:49.101606 31023 raft_consensus.cc:729] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a1f275c6b44657ae66f48ca7021be1, State: Initialized, Role: FOLLOWER
I20260807 22:42:49.101711 31023 consensus_queue.cc:260] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.101934 31023 ts_tablet_manager.cc:1428] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:42:49.102829 31024 tablet_bootstrap.cc:492] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6: No bootstrap required, opened a new log
I20260807 22:42:49.102912 31024 ts_tablet_manager.cc:1397] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:42:49.103309 31024 raft_consensus.cc:348] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.103413 31024 raft_consensus.cc:374] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:49.103444 31024 raft_consensus.cc:729] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88df149af36419094c2429ce10f30f6, State: Initialized, Role: FOLLOWER
I20260807 22:42:49.103554 31024 consensus_queue.cc:260] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.103799 31024 ts_tablet_manager.cc:1428] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:42:49.230135 30985 raft_consensus.cc:482] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:42:49.230268 30985 raft_consensus.cc:504] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.230719 30985 leader_election.cc:288] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771), 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:49.231163 30490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5356107706c3414dabfc4698cf029670" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64" is_pre_election: true
I20260807 22:42:49.231215 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5356107706c3414dabfc4698cf029670" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1" is_pre_election: true
I20260807 22:42:49.231309 30490 raft_consensus.cc:2454] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 0.
I20260807 22:42:49.231376 30723 raft_consensus.cc:2454] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 0.
I20260807 22:42:49.231616 30149 leader_election.cc:302] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88df149af36419094c2429ce10f30f6, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:49.231791 30985 raft_consensus.cc:2781] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:42:49.231848 30985 raft_consensus.cc:482] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:42:49.231894 30985 raft_consensus.cc:3037] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:49.232849 30985 raft_consensus.cc:504] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.233151 30985 leader_election.cc:288] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 election: Requested vote from peers d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771), 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875)
I20260807 22:42:49.233454 30490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5356107706c3414dabfc4698cf029670" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a13898824148e98fb6c55d6c487a64"
I20260807 22:42:49.233508 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5356107706c3414dabfc4698cf029670" candidate_uuid: "a88df149af36419094c2429ce10f30f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a1f275c6b44657ae66f48ca7021be1"
I20260807 22:42:49.233572 30490 raft_consensus.cc:3037] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:49.233618 30723 raft_consensus.cc:3037] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:49.234575 30490 raft_consensus.cc:2454] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 1.
I20260807 22:42:49.234745 30723 raft_consensus.cc:2454] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88df149af36419094c2429ce10f30f6 in term 1.
I20260807 22:42:49.234874 30149 leader_election.cc:302] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88df149af36419094c2429ce10f30f6, d9a13898824148e98fb6c55d6c487a64; no voters:
I20260807 22:42:49.235085 30985 raft_consensus.cc:2781] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:49.235183 30985 raft_consensus.cc:686] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [term 1 LEADER]: Becoming Leader. State: Replica: a88df149af36419094c2429ce10f30f6, State: Running, Role: LEADER
I20260807 22:42:49.235304 30985 consensus_queue.cc:237] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } }
I20260807 22:42:49.236951 29998 catalog_manager.cc:5495] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a88df149af36419094c2429ce10f30f6 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "a88df149af36419094c2429ce10f30f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88df149af36419094c2429ce10f30f6" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 } health_report { overall_health: UNKNOWN } } }
I20260807 22:42:49.257496 30723 raft_consensus.cc:1261] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Refusing update from remote peer a88df149af36419094c2429ce10f30f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:49.257524 30490 raft_consensus.cc:1261] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Refusing update from remote peer a88df149af36419094c2429ce10f30f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:42:49.257850 30985 consensus_queue.cc:1035] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a13898824148e98fb6c55d6c487a64" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 34771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:42:49.258044 30969 consensus_queue.cc:1035] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a1f275c6b44657ae66f48ca7021be1" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:42:49.269603 30970 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:42:50.817461 29998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:50.817629 29998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:42:50.819809 29998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3169bfbdb12f4c3899998b21c118acad: Sending DeleteTablet for 3 replicas of tablet 5356107706c3414dabfc4698cf029670
I20260807 22:42:50.820335 30457 tablet_service.cc:1514] Processing DeleteTablet for tablet 5356107706c3414dabfc4698cf029670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:50 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60948
I20260807 22:42:50.820415 30678 tablet_service.cc:1514] Processing DeleteTablet for tablet 5356107706c3414dabfc4698cf029670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:50 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:47012
I20260807 22:42:50.820400 30223 tablet_service.cc:1514] Processing DeleteTablet for tablet 5356107706c3414dabfc4698cf029670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:42:50 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:42078
I20260807 22:42:50.820783 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 30117
I20260807 22:42:50.827704 31090 tablet_replica.cc:330] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: stopping tablet replica
I20260807 22:42:50.827880 31090 raft_consensus.cc:2229] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:50.828044 31090 raft_consensus.cc:2258] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:50.828329 31092 tablet_replica.cc:330] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: stopping tablet replica
I20260807 22:42:50.828469 31092 raft_consensus.cc:2229] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:42:50.828594 31092 raft_consensus.cc:2258] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:42:50.832445 31092 ts_tablet_manager.cc:1905] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:50.834867 31092 ts_tablet_manager.cc:1918] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:42:50.834939 31092 log.cc:1196] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/5356107706c3414dabfc4698cf029670
I20260807 22:42:50.835155 31092 ts_tablet_manager.cc:1939] T 5356107706c3414dabfc4698cf029670 P 90a1f275c6b44657ae66f48ca7021be1: Deleting consensus metadata
I20260807 22:42:50.845775 29976 catalog_manager.cc:4841] TS 90a1f275c6b44657ae66f48ca7021be1 (127.26.189.195:36875): tablet 5356107706c3414dabfc4698cf029670 (table test-table [id=441fc8aafa9c40069c3927304f3af26c]) successfully deleted
I20260807 22:42:50.820716 31091 tablet_replica.cc:330] T 5356107706c3414dabfc4698cf029670 P a88df149af36419094c2429ce10f30f6: stopping tablet replica
I20260807 22:42:50.848055 31090 ts_tablet_manager.cc:1905] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:42:50.850395 31090 ts_tablet_manager.cc:1918] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:42:50.850488 31090 log.cc:1196] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/5356107706c3414dabfc4698cf029670
I20260807 22:42:50.850713 31090 ts_tablet_manager.cc:1939] T 5356107706c3414dabfc4698cf029670 P d9a13898824148e98fb6c55d6c487a64: Deleting consensus metadata
I20260807 22:42:50.851456 29974 catalog_manager.cc:4841] TS d9a13898824148e98fb6c55d6c487a64 (127.26.189.194:34771): tablet 5356107706c3414dabfc4698cf029670 (table test-table [id=441fc8aafa9c40069c3927304f3af26c]) successfully deleted
W20260807 22:42:50.854717 29973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:43939 (error 108)
I20260807 22:42:50.854817 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 30350
I20260807 22:42:50.861095 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:50.859337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:43939 (local address 127.0.0.1:41980)
0807 22:42:50.860925 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:50.861002 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1505,"thread_start_us":1401,"threads_started":1}
W20260807 22:42:50.861222 29973 catalog_manager.cc:4584] TS a88df149af36419094c2429ce10f30f6 (127.26.189.193:43939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5356107706c3414dabfc4698cf029670: Network error: Client connection negotiation failed: client connection to 127.26.189.193:43939: connect: Connection refused (error 111)
I20260807 22:42:50.867991 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 30588
I20260807 22:42:50.880126 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29949
2026-08-07T22:42:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6327 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:50 dist-test-slave-4tdj krb5kdc[31099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:50 dist-test-slave-4tdj krb5kdc[31099](info): set up 2 sockets
Aug 07 22:42:50 dist-test-slave-4tdj krb5kdc[31099](info): commencing operation
krb5kdc: starting...
W20260807 22:42:52.043920 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:52.163183 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:42495
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:42495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:52.270493 31120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:52.270736 31120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:52.270773 31120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:52.274643 31120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:52.274694 31120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:52.274712 31120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:52.274726 31120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:52.274746 31120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:52.279858 31120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:42495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:42495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:52.280795 31120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:52.281719 31120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:42:52.289755 31120 server_base.cc:1034] running on GCE node
W20260807 22:42:52.289777 31133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:52.289804 31130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:52.289857 31128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:52.290597 31120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:52.291646 31120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:52.292804 31120 hybrid_clock.cc:648] HybridClock initialized: now 1786142572292790 us; error 26 us; skew 500 ppm
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:52.295492 31120 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:52.296336 31120 webserver.cc:460] Webserver started at http://127.26.189.254:36753/ using document root <none> and password file <none>
I20260807 22:42:52.296845 31120 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:52.296902 31120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:52.297066 31120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:52.299074 31120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "4dbd6032a92a43aabf2ad67d337fa5b5"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "bff7dc3e3670c533583609fd6e1648e6"
server_key_iv: "7aeeec47fe9e49133f0cd4220b97da52"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.299578 31120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "4dbd6032a92a43aabf2ad67d337fa5b5"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "bff7dc3e3670c533583609fd6e1648e6"
server_key_iv: "7aeeec47fe9e49133f0cd4220b97da52"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.303087 31120 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:52.305310 31145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.306515 31120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:52.306627 31120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "4dbd6032a92a43aabf2ad67d337fa5b5"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "bff7dc3e3670c533583609fd6e1648e6"
server_key_iv: "7aeeec47fe9e49133f0cd4220b97da52"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.306711 31120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:52.327479 31120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:52.330466 31120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:52.330618 31120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:52.337304 31120 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:42495
I20260807 22:42:52.337320 31223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:42495 every 8 connection(s)
I20260807 22:42:52.338238 31120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:52.339011 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 31120
I20260807 22:42:52.339102 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:52.339365 27383 external_mini_cluster.cc:1428] Setting key 95ddf6141c5aef19721c23d7443c62cc
I20260807 22:42:52.341244 31225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:52.346585 31225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5: Bootstrap starting.
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:52.348781 31225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:52.349502 31225 log.cc:824] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:52.351289 31225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5: No bootstrap required, opened a new log
I20260807 22:42:52.352248 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:52.340684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.26.189.254:42495)
0807 22:42:52.341120 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:52.341136 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:52.341172 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:42:52.341804 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:52.341820 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:52.342133 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:52.342465 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:52.343239 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.344207 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.345024 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.345495 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.345635 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:52.347982 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:52.347987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:52.347999 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:52.348026 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:52.349635 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:52.350025 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:52.350031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:52.350036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:52.350151 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:52.350435 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:52.350439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:52.350440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:52.350816 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:52.350998 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:52.351262 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 22:42:52.351559 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":141,"threads_started":1}
I20260807 22:42:52.354179 31225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42495 } }
I20260807 22:42:52.354418 31225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:52.354501 31225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbd6032a92a43aabf2ad67d337fa5b5, State: Initialized, Role: FOLLOWER
I20260807 22:42:52.354883 31225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42495 } }
I20260807 22:42:52.355005 31225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:52.355069 31225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:52.355171 31225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:52.355973 31225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42495 } }
I20260807 22:42:52.356276 31225 leader_election.cc:302] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dbd6032a92a43aabf2ad67d337fa5b5; no voters:
I20260807 22:42:52.356544 31225 leader_election.cc:288] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:52.356631 31234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:52.356876 31234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbd6032a92a43aabf2ad67d337fa5b5, State: Running, Role: LEADER
I20260807 22:42:52.357219 31234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42495 } }
I20260807 22:42:52.357743 31225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:52.359553 31238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dbd6032a92a43aabf2ad67d337fa5b5. Latest consensus state: current_term: 1 leader_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42495 } } }
I20260807 22:42:52.359671 31238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:52.360355 31235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dbd6032a92a43aabf2ad67d337fa5b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42495 } } }
I20260807 22:42:52.360740 31235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5 [sys.catalog]: This master's current role is: LEADER
W20260807 22:42:52.362538 31259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:42:52.362628 31259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:42:52.362696 31260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:52.363399 31260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:52.368050 31260 catalog_manager.cc:1347] Generated new cluster ID: 76ccd1270a4a449a8556309a67f5e995
I20260807 22:42:52.368112 31260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:52.394379 31260 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:52.395277 31260 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:52.399430 31260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dbd6032a92a43aabf2ad67d337fa5b5: Generated new TSK 0
I20260807 22:42:52.400210 31260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:52.435714 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42495
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:52.547607 31269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:52.547855 31269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:52.547902 31269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:52.552206 31269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:52.552268 31269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:52.552349 31269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:52.557435 31269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:52.558487 31269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:52.559532 31269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:52.568799 31281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:52.568833 31279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:52.568797 31283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:52.568985 31269 server_base.cc:1034] running on GCE node
I20260807 22:42:52.569365 31269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:52.569958 31269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:52.571127 31269 hybrid_clock.cc:648] HybridClock initialized: now 1786142572571114 us; error 27 us; skew 500 ppm
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:52.573990 31269 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:52.574863 31269 webserver.cc:460] Webserver started at http://127.26.189.193:34239/ using document root <none> and password file <none>
I20260807 22:42:52.575394 31269 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:52.575461 31269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:52.575661 31269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:52.577657 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "0460a944a4fd401b8324cc90de5e13e7"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "18fcfbaf03e18f4da1b3be9dc6d19b02"
server_key_iv: "e76da9db8b671dd4b2fc02ce82174da9"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.578194 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "0460a944a4fd401b8324cc90de5e13e7"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "18fcfbaf03e18f4da1b3be9dc6d19b02"
server_key_iv: "e76da9db8b671dd4b2fc02ce82174da9"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.581631 31269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:42:52.583910 31291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.585013 31269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:52.585117 31269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "0460a944a4fd401b8324cc90de5e13e7"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "18fcfbaf03e18f4da1b3be9dc6d19b02"
server_key_iv: "e76da9db8b671dd4b2fc02ce82174da9"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.585237 31269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:52.630388 31269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:52.633023 31269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:52.633205 31269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:52.633780 31269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:52.634610 31269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:52.634658 31269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.634711 31269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:52.634740 31269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.643196 31269 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:37543
I20260807 22:42:52.643220 31473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:37543 every 8 connection(s)
I20260807 22:42:52.644037 31269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:52.652493 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 31269
I20260807 22:42:52.652631 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:52.652922 27383 external_mini_cluster.cc:1428] Setting key 32d6d18529cba5678b9994b7ecfbb128
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:52.657294 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:52.645741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:36913 (local address 127.26.189.254:42495)
0807 22:42:52.645873 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:52.645876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:52.646700 (+ 824us) server_negotiation.cc:407] Connection header received
0807 22:42:52.647922 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:52.647925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:52.647974 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:52.648046 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:52.649167 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.649703 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.650926 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.651161 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.651251 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:52.653777 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:52.653780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:52.653783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:52.653806 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:52.655207 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:52.655751 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:52.655754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:52.655755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:52.655828 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:52.656173 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:52.656176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:52.656178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:52.656406 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:52.656508 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:52.656953 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 22:42:52.657083 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:42:52.658100 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:52.646091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42495 (local address 127.26.189.193:36913)
0807 22:42:52.646554 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:52.646595 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:52.647733 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:52.648194 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:52.648200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:52.648541 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:52.649029 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:52.649043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.649831 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.649834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:52.650783 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:52.650799 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.651290 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.651293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:52.651432 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:52.651990 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:52.652007 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:52.653632 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:52.655324 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:52.655330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:52.655345 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:52.655661 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:52.655899 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:52.655902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:52.655903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:52.656080 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:52.656509 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:52.656513 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:52.656739 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 22:42:52.656890 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:42:52.657145 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":138,"threads_started":1}
I20260807 22:42:52.659338 31474 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42495
I20260807 22:42:52.659596 31474 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:52.660192 31474 heartbeater.cc:511] Master 127.26.189.254:42495 requested a full tablet report, sending...
I20260807 22:42:52.661670 31171 ts_manager.cc:194] Registered new tserver with Master: 0460a944a4fd401b8324cc90de5e13e7 (127.26.189.193:37543)
I20260807 22:42:52.662817 31171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:36913
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:52.679131 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42495
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:52.786687 31482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:52.786906 31482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:52.786943 31482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:52.790881 31482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:52.790928 31482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:52.790989 31482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:52.795967 31482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:52.796885 31482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:52.797834 31482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:52.806882 31493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:52.806945 31497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:52.806883 31500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:52.807205 31482 server_base.cc:1034] running on GCE node
I20260807 22:42:52.807561 31482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:52.808188 31482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:52.809346 31482 hybrid_clock.cc:648] HybridClock initialized: now 1786142572809325 us; error 35 us; skew 500 ppm
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:52.811967 31482 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:52.812765 31482 webserver.cc:460] Webserver started at http://127.26.189.194:46169/ using document root <none> and password file <none>
I20260807 22:42:52.813335 31482 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:52.813396 31482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:52.813561 31482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:52.815646 31482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "e786c90c4a1f4b21a7a746a35bef83a2"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "596eb02eeb577b6d4ce7bba73a91e975"
server_key_iv: "b75a882e74c7c0a8986307d102ac2f80"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.816179 31482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "e786c90c4a1f4b21a7a746a35bef83a2"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "596eb02eeb577b6d4ce7bba73a91e975"
server_key_iv: "b75a882e74c7c0a8986307d102ac2f80"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.819758 31482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:42:52.822362 31517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.823552 31482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:52.823652 31482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "e786c90c4a1f4b21a7a746a35bef83a2"
format_stamp: "Formatted at 2026-08-07 22:42:52 on dist-test-slave-4tdj"
server_key: "596eb02eeb577b6d4ce7bba73a91e975"
server_key_iv: "b75a882e74c7c0a8986307d102ac2f80"
server_key_version: "encryptionkey@0"
I20260807 22:42:52.823737 31482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:52.841758 31482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:52.844372 31482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:52.844547 31482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:52.845086 31482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:52.845876 31482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:52.845922 31482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.845960 31482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:52.845980 31482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:52.854398 31482 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:44061
I20260807 22:42:52.854429 31720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:44061 every 8 connection(s)
I20260807 22:42:52.855244 31482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:42:52 dist-test-slave-4tdj krb5kdc[31099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142572, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:52.864889 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 31482
I20260807 22:42:52.865001 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:52.865294 27383 external_mini_cluster.cc:1428] Setting key 73449a04c17d514766cd918d10bbc35f
I20260807 22:42:52.868080 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:52.856908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:43191 (local address 127.26.189.254:42495)
0807 22:42:52.857066 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:52.857071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:52.858324 (+ 1253us) server_negotiation.cc:407] Connection header received
0807 22:42:52.859210 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:52.859213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:52.859260 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:52.859329 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:52.860435 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.860991 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.862110 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.862361 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.862434 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:52.864662 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:52.864667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:52.864669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:52.864698 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:52.866095 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:52.866630 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:52.866633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:52.866635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:52.866731 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:52.867100 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:52.867102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:52.867103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:52.867268 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:52.867340 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:52.867876 (+ 536us) server_negotiation.cc:299] Negotiation successful
0807 22:42:52.867968 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:42:52.869073 31727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:52.857292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42495 (local address 127.26.189.194:43191)
0807 22:42:52.858138 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:52.858196 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:52.859040 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:52.859485 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:52.859523 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:52.859855 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:52.860302 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:52.860315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.861114 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.861117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:52.862001 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:52.862011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:52.862485 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:52.862488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:52.862639 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:52.863156 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:52.863171 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:52.864485 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:52.866199 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:52.866204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:52.866215 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:52.866525 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:52.866827 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:52.866830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:52.866832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:52.867008 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:52.867362 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:52.867367 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:52.867659 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 22:42:52.867854 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 22:42:52.868153 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":136,"threads_started":1}
I20260807 22:42:52.870118 31722 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42495
I20260807 22:42:52.870370 31722 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:52.870886 31722 heartbeater.cc:511] Master 127.26.189.254:42495 requested a full tablet report, sending...
I20260807 22:42:52.871925 31171 ts_manager.cc:194] Registered new tserver with Master: e786c90c4a1f4b21a7a746a35bef83a2 (127.26.189.194:44061)
I20260807 22:42:52.872458 31171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:43191
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:52.893155 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42495
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:53.001099 31733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:53.001336 31733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:53.001394 31733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:53.005302 31733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:53.005357 31733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:53.005431 31733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:53.010594 31733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:53.011518 31733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:53.012465 31733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:53.022218 31742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:53.022317 31743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:53.022195 31748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:53.022686 31733 server_base.cc:1034] running on GCE node
I20260807 22:42:53.023211 31733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:53.023991 31733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:53.025148 31733 hybrid_clock.cc:648] HybridClock initialized: now 1786142573025123 us; error 45 us; skew 500 ppm
Aug 07 22:42:53 dist-test-slave-4tdj krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142573, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:53.028308 31733 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:53.029150 31733 webserver.cc:460] Webserver started at http://127.26.189.195:37147/ using document root <none> and password file <none>
I20260807 22:42:53.029665 31733 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:53.029732 31733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:53.029928 31733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:53.031930 31733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "5edcd08fb5764397a37977235aa25bfe"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-4tdj"
server_key: "41f2bf47afc3836584fd0c9f2bbc2351"
server_key_iv: "b3902278f1d023ef8546afed2555ba0a"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.032452 31733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "5edcd08fb5764397a37977235aa25bfe"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-4tdj"
server_key: "41f2bf47afc3836584fd0c9f2bbc2351"
server_key_iv: "b3902278f1d023ef8546afed2555ba0a"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.035914 31733 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:42:53.038188 31756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:53.039371 31733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:53.039475 31733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "5edcd08fb5764397a37977235aa25bfe"
format_stamp: "Formatted at 2026-08-07 22:42:53 on dist-test-slave-4tdj"
server_key: "41f2bf47afc3836584fd0c9f2bbc2351"
server_key_iv: "b3902278f1d023ef8546afed2555ba0a"
server_key_version: "encryptionkey@0"
I20260807 22:42:53.039563 31733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:53.057596 31733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:53.060799 31733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:53.060972 31733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:53.061566 31733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:53.062436 31733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:53.062485 31733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:53.062538 31733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:53.062568 31733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:53.070874 31733 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:35171
I20260807 22:42:53.070967 31932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:35171 every 8 connection(s)
I20260807 22:42:53.071792 31733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:53.079094 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 31733
I20260807 22:42:53.079197 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:53.079479 27383 external_mini_cluster.cc:1428] Setting key 6bd8956d85e9a94faed726b50196097b
Aug 07 22:42:53 dist-test-slave-4tdj krb5kdc[31099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142573, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:53.084529 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:53.073575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:39053 (local address 127.26.189.254:42495)
0807 22:42:53.073702 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:53.073706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:53.074787 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 22:42:53.075673 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:53.075677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:53.075735 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:53.075814 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:53.076944 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:53.077475 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:53.078636 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:53.078877 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:53.078971 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:53.081589 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:53.081593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:53.081596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:53.081624 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:53.082848 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:53.083312 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:53.083315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:53.083316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:53.083390 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:53.083675 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:53.083678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:53.083679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:53.083850 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:53.083925 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:53.084319 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 22:42:53.084412 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:42:53.085318 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:53.073900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42495 (local address 127.26.189.195:39053)
0807 22:42:53.074633 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:53.074693 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:53.075467 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:53.075985 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:53.075992 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:53.076333 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:53.076802 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:53.076815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:53.077597 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:53.077600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:53.078506 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:53.078524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:53.079023 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:53.079027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:53.079205 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:53.079815 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:53.079831 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:53.081414 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:53.082968 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:53.082972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:53.082983 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:53.083230 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:53.083462 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:53.083464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:53.083465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:53.083585 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:53.083924 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:53.083928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:53.084142 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 22:42:53.084289 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:42:53.084520 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":120,"threads_started":1}
I20260807 22:42:53.086277 31933 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42495
I20260807 22:42:53.086514 31933 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:53.087018 31933 heartbeater.cc:511] Master 127.26.189.254:42495 requested a full tablet report, sending...
I20260807 22:42:53.087968 31171 ts_manager.cc:194] Registered new tserver with Master: 5edcd08fb5764397a37977235aa25bfe (127.26.189.195:35171)
I20260807 22:42:53.088557 31171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:39053
I20260807 22:42:53.093003 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:42:53.107179 31229 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:42:53.105799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.26.189.254:42495)
0807 22:42:53.105934 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:53.105938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:53.106114 (+ 176us) server_negotiation.cc:407] Connection header received
0807 22:42:53.106223 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:53.106226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:53.106266 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:53.106371 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:53.107067 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52250: BlockingRecv error: recv got EOF from 127.0.0.1:52250 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:42:53.108390 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 31269
I20260807 22:42:53.121047 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 31482
I20260807 22:42:53.129276 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 31733
I20260807 22:42:53.136538 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 31120
2026-08-07T22:42:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2256 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:53 dist-test-slave-4tdj krb5kdc[31949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:53 dist-test-slave-4tdj krb5kdc[31949](info): set up 2 sockets
Aug 07 22:42:53 dist-test-slave-4tdj krb5kdc[31949](info): commencing operation
krb5kdc: starting...
W20260807 22:42:54.271970 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:54 dist-test-slave-4tdj krb5kdc[31949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:54.384955 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39889
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:54.492466 31967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:54.492713 31967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:54.492769 31967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:54.496652 31967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:54.496702 31967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:54.496728 31967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:54.496747 31967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:54.496773 31967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:54.501732 31967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:54.502694 31967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:54.503654 31967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:54.511871 31977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:54.511915 31983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:54.511916 31967 server_base.cc:1034] running on GCE node
W20260807 22:42:54.511871 31980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:54.512794 31967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:54.513866 31967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:54.515046 31967 hybrid_clock.cc:648] HybridClock initialized: now 1786142574515029 us; error 30 us; skew 500 ppm
Aug 07 22:42:54 dist-test-slave-4tdj krb5kdc[31949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:54.521934 31967 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:54.522876 31967 webserver.cc:460] Webserver started at http://127.26.189.254:36041/ using document root <none> and password file <none>
I20260807 22:42:54.523420 31967 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:54.523488 31967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:54.523702 31967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:54.525763 31967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "fe920148e8cf4e458fc3affbc3bb4297"
format_stamp: "Formatted at 2026-08-07 22:42:54 on dist-test-slave-4tdj"
server_key: "079beec922b802ca2c56d68dd958a707"
server_key_iv: "3f72737af1bf334047ca71ee5e0d4dc8"
server_key_version: "encryptionkey@0"
I20260807 22:42:54.526363 31967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "fe920148e8cf4e458fc3affbc3bb4297"
format_stamp: "Formatted at 2026-08-07 22:42:54 on dist-test-slave-4tdj"
server_key: "079beec922b802ca2c56d68dd958a707"
server_key_iv: "3f72737af1bf334047ca71ee5e0d4dc8"
server_key_version: "encryptionkey@0"
I20260807 22:42:54.529992 31967 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:54.532395 31993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:54.533609 31967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:42:54.533727 31967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "fe920148e8cf4e458fc3affbc3bb4297"
format_stamp: "Formatted at 2026-08-07 22:42:54 on dist-test-slave-4tdj"
server_key: "079beec922b802ca2c56d68dd958a707"
server_key_iv: "3f72737af1bf334047ca71ee5e0d4dc8"
server_key_version: "encryptionkey@0"
I20260807 22:42:54.533823 31967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:54.560415 31967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:54.563212 31967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:54.563395 31967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:54.570204 31967 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:39889
I20260807 22:42:54.570472 32080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:39889 every 8 connection(s)
I20260807 22:42:54.571177 31967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:54.573865 32081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:54.579285 32081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297: Bootstrap starting.
I20260807 22:42:54.580916 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 31967
I20260807 22:42:54.581044 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:54.581324 27383 external_mini_cluster.cc:1428] Setting key 2db1c4e3089228e0067cfca7f3728d2d
I20260807 22:42:54.581898 32081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:54.582795 32081 log.cc:824] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:54.584568 32081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297: No bootstrap required, opened a new log
I20260807 22:42:54.587909 32081 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe920148e8cf4e458fc3affbc3bb4297" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39889 } }
I20260807 22:42:54.588122 32081 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:54.588188 32081 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe920148e8cf4e458fc3affbc3bb4297, State: Initialized, Role: FOLLOWER
I20260807 22:42:54.588594 32081 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe920148e8cf4e458fc3affbc3bb4297" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39889 } }
I20260807 22:42:54.588725 32081 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:54.588789 32081 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:54.588882 32081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 22:42:54 dist-test-slave-4tdj krb5kdc[31949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:54.589954 32081 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe920148e8cf4e458fc3affbc3bb4297" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39889 } }
I20260807 22:42:54.590328 32081 leader_election.cc:302] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe920148e8cf4e458fc3affbc3bb4297; no voters:
I20260807 22:42:54.590646 32081 leader_election.cc:288] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:54.590744 32093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:54.590981 32093 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [term 1 LEADER]: Becoming Leader. State: Replica: fe920148e8cf4e458fc3affbc3bb4297, State: Running, Role: LEADER
I20260807 22:42:54.591352 32093 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe920148e8cf4e458fc3affbc3bb4297" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39889 } }
I20260807 22:42:54.591881 32081 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:54.593077 32097 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe920148e8cf4e458fc3affbc3bb4297. Latest consensus state: current_term: 1 leader_uuid: "fe920148e8cf4e458fc3affbc3bb4297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe920148e8cf4e458fc3affbc3bb4297" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39889 } } }
I20260807 22:42:54.593197 32097 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:54.593719 32107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:54.594017 32096 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe920148e8cf4e458fc3affbc3bb4297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe920148e8cf4e458fc3affbc3bb4297" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39889 } } }
I20260807 22:42:54.594173 32096 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:54.595886 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.582446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.26.189.254:39889)
0807 22:42:54.582894 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:54.582911 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:54.582954 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:42:54.583581 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:54.583598 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:54.583902 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:54.584211 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:54.585101 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.586094 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.586972 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.587434 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.587589 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.590225 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:54.590230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:54.590244 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:54.590270 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:54.592353 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.592911 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.592917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.592922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.593070 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:54.593384 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:54.593387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:54.593389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:54.593942 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:54.594147 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:54.594934 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 22:42:54.595180 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":142,"threads_started":1}
I20260807 22:42:54.597183 32107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:54.598593 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:42:54.602885 32107 catalog_manager.cc:1347] Generated new cluster ID: 1f3eb9e0f2f748099bd91c74aacd2e02
I20260807 22:42:54.602969 32107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:54.610185 32107 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:54.611393 32107 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:54.622335 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:54.617729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.26.189.254:39889)
0807 22:42:54.617845 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:54.617851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:54.618100 (+ 249us) server_negotiation.cc:407] Connection header received
0807 22:42:54.618228 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:54.618231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:54.618277 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:54.618372 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:54.618909 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.619428 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.620289 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:54.620539 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:54.620628 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:54.621404 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:54.621408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:54.621410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:42:54.621863 (+ 453us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:54.621894 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:42:54.621910 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:54.621993 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:54.622106 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 22:42:54.622173 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:42:54.625360 32107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe920148e8cf4e458fc3affbc3bb4297: Generated new TSK 0
I20260807 22:42:54.626130 32107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:42:54.633665 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 31967
2026-08-07T22:42:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1490 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:54 dist-test-slave-4tdj krb5kdc[32133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:54 dist-test-slave-4tdj krb5kdc[32133](info): set up 2 sockets
Aug 07 22:42:54 dist-test-slave-4tdj krb5kdc[32133](info): commencing operation
krb5kdc: starting...
W20260807 22:42:55.766947 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:55 dist-test-slave-4tdj krb5kdc[32133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:55.886466 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:46279
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:55.995126 32153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:55.995384 32153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:55.995440 32153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:55.999425 32153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:55.999488 32153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:55.999514 32153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:55.999538 32153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:55.999568 32153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:56.004788 32153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:46279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:56.005836 32153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:56.006865 32153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:56.015094 32165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.015101 32161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.015108 32167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:56.015326 32153 server_base.cc:1034] running on GCE node
I20260807 22:42:56.015936 32153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:56.017012 32153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:56.018174 32153 hybrid_clock.cc:648] HybridClock initialized: now 1786142576018150 us; error 41 us; skew 500 ppm
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:56.021024 32153 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:56.021994 32153 webserver.cc:460] Webserver started at http://127.26.189.254:42347/ using document root <none> and password file <none>
I20260807 22:42:56.022617 32153 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:56.022697 32153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:56.022917 32153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:56.025000 32153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "9865f6b2ebb740b58bf669929a3b91f2"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "052205d5389409b5837de23d63a19324"
server_key_iv: "ddac7b0e9d1bb6031367639cc61ef5c6"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.025590 32153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "9865f6b2ebb740b58bf669929a3b91f2"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "052205d5389409b5837de23d63a19324"
server_key_iv: "ddac7b0e9d1bb6031367639cc61ef5c6"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.029241 32153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:42:56.031764 32178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.032984 32153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:56.033106 32153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "9865f6b2ebb740b58bf669929a3b91f2"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "052205d5389409b5837de23d63a19324"
server_key_iv: "ddac7b0e9d1bb6031367639cc61ef5c6"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.033195 32153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:56.052115 32153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:56.054977 32153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:56.055147 32153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:56.062160 32153 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:46279
I20260807 22:42:56.062443 32265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:46279 every 8 connection(s)
I20260807 22:42:56.063186 32153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:56.065867 32266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:56.071296 32266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2: Bootstrap starting.
I20260807 22:42:56.072242 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 32153
I20260807 22:42:56.072371 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:56.072667 27383 external_mini_cluster.cc:1428] Setting key 2f082fff12be239fa957c817498bb90e
I20260807 22:42:56.074128 32266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:56.074853 32266 log.cc:824] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:56.076943 32266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2: No bootstrap required, opened a new log
I20260807 22:42:56.080961 32266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9865f6b2ebb740b58bf669929a3b91f2" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46279 } }
I20260807 22:42:56.081224 32266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:56.081286 32266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9865f6b2ebb740b58bf669929a3b91f2, State: Initialized, Role: FOLLOWER
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:56.081681 32266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9865f6b2ebb740b58bf669929a3b91f2" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46279 } }
I20260807 22:42:56.081802 32266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:56.081866 32266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:42:56.081962 32266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:56.083033 32266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9865f6b2ebb740b58bf669929a3b91f2" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46279 } }
I20260807 22:42:56.083392 32266 leader_election.cc:302] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9865f6b2ebb740b58bf669929a3b91f2; no voters:
I20260807 22:42:56.083740 32266 leader_election.cc:288] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:56.083830 32273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:56.084045 32273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [term 1 LEADER]: Becoming Leader. State: Replica: 9865f6b2ebb740b58bf669929a3b91f2, State: Running, Role: LEADER
I20260807 22:42:56.084388 32273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9865f6b2ebb740b58bf669929a3b91f2" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46279 } }
I20260807 22:42:56.085009 32266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:56.086418 32275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9865f6b2ebb740b58bf669929a3b91f2. Latest consensus state: current_term: 1 leader_uuid: "9865f6b2ebb740b58bf669929a3b91f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9865f6b2ebb740b58bf669929a3b91f2" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46279 } } }
I20260807 22:42:56.086429 32274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9865f6b2ebb740b58bf669929a3b91f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9865f6b2ebb740b58bf669929a3b91f2" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46279 } } }
I20260807 22:42:56.086548 32274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:56.086547 32275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:56.087018 32285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:56.088248 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.073808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.26.189.254:46279)
0807 22:42:56.074215 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.074236 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.074279 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:42:56.074931 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.074948 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.075359 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.075702 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.076587 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.077631 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.078844 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.079313 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.079431 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.082341 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.082348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.082363 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.082395 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.084372 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.084841 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.084847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.084852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.084992 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.085306 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.085310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.085311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.085844 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.086042 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.086926 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.087253 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":129,"threads_started":1}
I20260807 22:42:56.090322 32285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:56.095894 32285 catalog_manager.cc:1347] Generated new cluster ID: f440b0d2e8ed495a878aefd13a27d59d
I20260807 22:42:56.095952 32285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:56.102664 32285 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:56.103534 32285 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:56.109503 32285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9865f6b2ebb740b58bf669929a3b91f2: Generated new TSK 0
I20260807 22:42:56.110069 32285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:56.141222 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46279
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:56.249401 32305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:56.249650 32305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:56.249711 32305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:56.253610 32305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:56.253669 32305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:56.253747 32305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:56.258890 32305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:56.259850 32305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:56.260820 32305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:56.270185 32322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.270210 32318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.270156 32317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:56.270342 32305 server_base.cc:1034] running on GCE node
I20260807 22:42:56.270843 32305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:56.271445 32305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:56.272595 32305 hybrid_clock.cc:648] HybridClock initialized: now 1786142576272584 us; error 28 us; skew 500 ppm
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:56.275233 32305 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:56.276026 32305 webserver.cc:460] Webserver started at http://127.26.189.193:34263/ using document root <none> and password file <none>
I20260807 22:42:56.276540 32305 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:56.276605 32305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:56.276793 32305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:56.278774 32305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "9b41d645a0cb45ec9f24c811044b79cd"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "6623dc407eb77dac832cd27a231aada8"
server_key_iv: "c634af8fa33cf50387a4fcd5db39d252"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.279296 32305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "9b41d645a0cb45ec9f24c811044b79cd"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "6623dc407eb77dac832cd27a231aada8"
server_key_iv: "c634af8fa33cf50387a4fcd5db39d252"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.282840 32305 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:56.285063 32329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.286422 32305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:56.286538 32305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "9b41d645a0cb45ec9f24c811044b79cd"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "6623dc407eb77dac832cd27a231aada8"
server_key_iv: "c634af8fa33cf50387a4fcd5db39d252"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.286646 32305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:56.296238 32305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:56.299297 32305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:56.299469 32305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:56.300037 32305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:56.300835 32305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:56.300884 32305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.300935 32305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:56.300963 32305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.309381 32305 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:43757
I20260807 22:42:56.309435 32537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:43757 every 8 connection(s)
I20260807 22:42:56.310391 32305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:56.317418 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 32305
I20260807 22:42:56.317550 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:56.317883 27383 external_mini_cluster.cc:1428] Setting key 4c09f66a549d5786a906f85009308782
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:56.323832 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.312126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:36067 (local address 127.26.189.254:46279)
0807 22:42:56.312268 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.312273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.313121 (+ 848us) server_negotiation.cc:407] Connection header received
0807 22:42:56.314447 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.314450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.314497 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.314566 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.315640 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.316220 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.317374 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.317731 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.317834 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.320374 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.320377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.320379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.320402 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.321847 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.322445 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.322448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.322449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.322524 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.322853 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.322854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.322856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.323015 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.323081 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.323616 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.323713 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:42:56.324961 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.312494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:46279 (local address 127.26.189.193:36067)
0807 22:42:56.312956 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:56.312999 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:56.314230 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:56.314694 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:56.314700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:56.315035 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:56.315520 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.315534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.316350 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.316353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.317251 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.317262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.317872 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.317875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.318039 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.318650 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:56.318666 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:56.320219 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:56.321958 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.321964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.321980 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.322356 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.322603 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.322607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.322609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.322774 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.323102 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:56.323108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:56.323395 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 22:42:56.323582 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 22:42:56.323914 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":124,"threads_started":1}
I20260807 22:42:56.326200 32539 heartbeater.cc:347] Connected to a master server at 127.26.189.254:46279
I20260807 22:42:56.326448 32539 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:56.326923 32539 heartbeater.cc:511] Master 127.26.189.254:46279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:42:56.328538 32212 ts_manager.cc:194] Registered new tserver with Master: 9b41d645a0cb45ec9f24c811044b79cd (127.26.189.193:43757)
I20260807 22:42:56.329680 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:36067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:56.347302 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46279
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:56.457506 32549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:56.457782 32549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:56.457839 32549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:56.462075 32549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:56.462132 32549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:56.462195 32549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:56.467314 32549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:56.468267 32549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:56.469211 32549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:56.478314 32562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.478351 32557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.478314 32558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:56.478652 32549 server_base.cc:1034] running on GCE node
I20260807 22:42:56.479043 32549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:56.479594 32549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:56.480748 32549 hybrid_clock.cc:648] HybridClock initialized: now 1786142576480736 us; error 29 us; skew 500 ppm
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:56.483352 32549 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:56.484187 32549 webserver.cc:460] Webserver started at http://127.26.189.194:43747/ using document root <none> and password file <none>
I20260807 22:42:56.484694 32549 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:56.484752 32549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:56.484943 32549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:56.486987 32549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "fb686eaa98e748d983b6284ae0e1bd24"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "573a5d1f9a07b5b485804efa24561106"
server_key_iv: "73ac761f18b152804c3ab429d94720e1"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.487502 32549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "fb686eaa98e748d983b6284ae0e1bd24"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "573a5d1f9a07b5b485804efa24561106"
server_key_iv: "73ac761f18b152804c3ab429d94720e1"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.491079 32549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:42:56.493276 32576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.494460 32549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:56.494563 32549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "fb686eaa98e748d983b6284ae0e1bd24"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "573a5d1f9a07b5b485804efa24561106"
server_key_iv: "73ac761f18b152804c3ab429d94720e1"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.494657 32549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:56.510039 32549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:56.512977 32549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:56.513154 32549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:56.513725 32549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:56.514530 32549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:56.514587 32549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.514640 32549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:56.514665 32549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.523245 32549 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:38831
I20260807 22:42:56.523270 32767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:38831 every 8 connection(s)
I20260807 22:42:56.524070 32549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:56.534080 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 32549
I20260807 22:42:56.534190 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:56.534490 27383 external_mini_cluster.cc:1428] Setting key 7d107735b02d9f9eafaa64d00e7c3b2c
I20260807 22:42:56.536616 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.525776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:38187 (local address 127.26.189.254:46279)
0807 22:42:56.525904 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.525908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.526753 (+ 845us) server_negotiation.cc:407] Connection header received
0807 22:42:56.528072 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.528075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.528127 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.528201 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.529252 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.529765 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.530847 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.531057 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.531131 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.533324 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.533328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.533330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.533355 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.534695 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.535165 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.535169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.535171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.535262 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.535594 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.535596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.535597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.535795 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.535876 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.536359 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.536473 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:42:56.537401 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.526133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:46279 (local address 127.26.189.194:38187)
0807 22:42:56.526600 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:56.526656 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:56.527869 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:56.528344 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:56.528351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:56.528692 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:56.529137 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.529149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.529853 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.529855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.530756 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.530765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.531190 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.531194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.531361 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.531842 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:56.531856 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:56.533178 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:56.534800 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.534806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.534820 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.535069 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.535345 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.535347 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.535349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.535515 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.535897 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:56.535902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:56.536135 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:42:56.536280 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 22:42:56.536538 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":147,"threads_started":1}
I20260807 22:42:56.538647 300 heartbeater.cc:347] Connected to a master server at 127.26.189.254:46279
I20260807 22:42:56.538890 300 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:56.539381 300 heartbeater.cc:511] Master 127.26.189.254:46279 requested a full tablet report, sending...
I20260807 22:42:56.540350 32212 ts_manager.cc:194] Registered new tserver with Master: fb686eaa98e748d983b6284ae0e1bd24 (127.26.189.194:38831)
I20260807 22:42:56.540922 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:38187
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:56.560657 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46279
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:56.667558 306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:56.667788 306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:56.667831 306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:56.671690 306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:56.671737 306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:56.671803 306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:56.676764 306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:56.677667 306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:56.678577 306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:56.687529 311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:56.687561 312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:56.687737 306 server_base.cc:1034] running on GCE node
W20260807 22:42:56.687964 315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:56.688333 306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:56.688890 306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:56.690040 306 hybrid_clock.cc:648] HybridClock initialized: now 1786142576690026 us; error 28 us; skew 500 ppm
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:56.692682 306 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:56.693480 306 webserver.cc:460] Webserver started at http://127.26.189.195:39391/ using document root <none> and password file <none>
I20260807 22:42:56.693982 306 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:56.694046 306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:56.694224 306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:56.696144 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "dfdba5eb341e4bb5b14b2212f12daa39"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "d73426b005b574ca5dc3b1de32901641"
server_key_iv: "be748a4044ba600e47a4ca2f9ed907db"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.696664 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "dfdba5eb341e4bb5b14b2212f12daa39"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "d73426b005b574ca5dc3b1de32901641"
server_key_iv: "be748a4044ba600e47a4ca2f9ed907db"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.700141 306 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:42:56.702310 331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.703413 306 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:56.703529 306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "dfdba5eb341e4bb5b14b2212f12daa39"
format_stamp: "Formatted at 2026-08-07 22:42:56 on dist-test-slave-4tdj"
server_key: "d73426b005b574ca5dc3b1de32901641"
server_key_iv: "be748a4044ba600e47a4ca2f9ed907db"
server_key_version: "encryptionkey@0"
I20260807 22:42:56.703634 306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:56.718420 306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:56.721319 306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:56.721485 306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:56.722035 306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:56.722842 306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:56.722890 306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.722941 306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:56.722968 306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:56.731150 306 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:34395
I20260807 22:42:56.731185 516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:34395 every 8 connection(s)
I20260807 22:42:56.731972 306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:56.736428 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 306
I20260807 22:42:56.736531 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:56.736828 27383 external_mini_cluster.cc:1428] Setting key fd1e0c9a2f9f5ee077e99bf418ba3c6b
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:56.744554 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.733647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:37047 (local address 127.26.189.254:46279)
0807 22:42:56.733783 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.733787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.734590 (+ 803us) server_negotiation.cc:407] Connection header received
0807 22:42:56.735909 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.735912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.735969 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.736044 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.737340 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.738044 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.739259 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.739476 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.739554 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.741776 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.741779 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.741780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.741805 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.742986 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.743402 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.743404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.743406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.743473 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.743717 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.743719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.743720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.743873 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.743939 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.744351 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.744441 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:42:56.745307 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.733966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:46279 (local address 127.26.189.195:37047)
0807 22:42:56.734453 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:56.734498 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:56.735714 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:56.736237 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:56.736243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:56.736649 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:56.737159 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.737178 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.738149 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.738152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.739153 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:56.739164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.739603 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.739606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:56.739746 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.740258 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:56.740273 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:56.741628 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:56.743079 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.743084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.743097 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.743326 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.743529 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:56.743531 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:56.743532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:56.743655 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:56.743938 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:56.743942 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:56.744147 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 22:42:56.744288 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 22:42:56.744509 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":113,"threads_started":1}
I20260807 22:42:56.746279 518 heartbeater.cc:347] Connected to a master server at 127.26.189.254:46279
I20260807 22:42:56.746518 518 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:56.746934 518 heartbeater.cc:511] Master 127.26.189.254:46279 requested a full tablet report, sending...
I20260807 22:42:56.747831 32212 ts_manager.cc:194] Registered new tserver with Master: dfdba5eb341e4bb5b14b2212f12daa39 (127.26.189.195:34395)
I20260807 22:42:56.748373 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:37047
I20260807 22:42:56.750285 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[32133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142576, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:56.775363 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:56.767676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58956 (local address 127.26.189.254:46279)
0807 22:42:56.767811 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:56.767816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:56.767940 (+ 124us) server_negotiation.cc:407] Connection header received
0807 22:42:56.768045 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:56.768048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:56.768087 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:56.768153 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:56.768829 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.769375 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.770222 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:56.770507 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:56.770608 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:56.772768 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:56.772771 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:56.772773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:56.772797 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:56.773967 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.774339 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.774342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.774344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.774416 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:56.774629 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:56.774631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:56.774633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:56.774831 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:56.774923 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:56.775117 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 22:42:56.775207 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 22:42:56.775738 32212 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58956
I20260807 22:42:56.776543 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 32305
I20260807 22:42:56.784330 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 32549
I20260807 22:42:56.806667 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 306
I20260807 22:42:56.814939 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 32153
2026-08-07T22:42:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2183 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[540](info): set up 2 sockets
Aug 07 22:42:56 dist-test-slave-4tdj krb5kdc[540](info): commencing operation
krb5kdc: starting...
W20260807 22:42:57.942072 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:42:58.058907 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:42865
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:42865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:58.165306 561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:58.165541 561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:58.165596 561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:58.169486 561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:42:58.169538 561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:58.169564 561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:58.169584 561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:42:58.169610 561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:42:58.174618 561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:42865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:42865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:58.175521 561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:58.176422 561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:58.184186 570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.184206 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.184257 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.184391 561 server_base.cc:1034] running on GCE node
I20260807 22:42:58.184966 561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:58.185946 561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:58.187103 561 hybrid_clock.cc:648] HybridClock initialized: now 1786142578187081 us; error 31 us; skew 500 ppm
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:58.189605 561 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:42:58.190408 561 webserver.cc:460] Webserver started at http://127.26.189.254:39529/ using document root <none> and password file <none>
I20260807 22:42:58.190903 561 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:58.190968 561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:58.191151 561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:58.193023 561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "e5698f79a071484096aaec5716230ab3"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "755c2abb05382813c9588428632e5879"
server_key_iv: "52595294698bef34a19645d714c192f3"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.193537 561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "e5698f79a071484096aaec5716230ab3"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "755c2abb05382813c9588428632e5879"
server_key_iv: "52595294698bef34a19645d714c192f3"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.196920 561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:42:58.199066 580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.200146 561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:58.200266 561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "e5698f79a071484096aaec5716230ab3"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "755c2abb05382813c9588428632e5879"
server_key_iv: "52595294698bef34a19645d714c192f3"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.200353 561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:58.215164 561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:58.217767 561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:58.217933 561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:58.224550 561 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:42865
I20260807 22:42:58.224591 660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:42865 every 8 connection(s)
I20260807 22:42:58.225466 561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:42:58.228097 661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:42:58.233429 661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3: Bootstrap starting.
I20260807 22:42:58.234493 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 561
I20260807 22:42:58.234635 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:42:58.234927 27383 external_mini_cluster.cc:1428] Setting key 5f7600912f120239e372ae0249047253
I20260807 22:42:58.236195 661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3: Neither blocks nor log segments found. Creating new log.
I20260807 22:42:58.236903 661 log.cc:824] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:42:58.238759 661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3: No bootstrap required, opened a new log
I20260807 22:42:58.242143 661 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5698f79a071484096aaec5716230ab3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42865 } }
I20260807 22:42:58.242373 661 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:42:58.242439 661 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5698f79a071484096aaec5716230ab3, State: Initialized, Role: FOLLOWER
I20260807 22:42:58.242856 661 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5698f79a071484096aaec5716230ab3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42865 } }
I20260807 22:42:58.242987 661 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:42:58.243050 661 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:58.243156 661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:42:58.244163 661 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5698f79a071484096aaec5716230ab3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42865 } }
I20260807 22:42:58.244498 661 leader_election.cc:302] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5698f79a071484096aaec5716230ab3; no voters:
I20260807 22:42:58.244799 661 leader_election.cc:288] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:42:58.244903 669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:42:58.245112 669 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [term 1 LEADER]: Becoming Leader. State: Replica: e5698f79a071484096aaec5716230ab3, State: Running, Role: LEADER
I20260807 22:42:58.245489 669 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5698f79a071484096aaec5716230ab3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42865 } }
I20260807 22:42:58.245931 661 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:42:58.246614 669 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5698f79a071484096aaec5716230ab3. Latest consensus state: current_term: 1 leader_uuid: "e5698f79a071484096aaec5716230ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5698f79a071484096aaec5716230ab3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42865 } } }
I20260807 22:42:58.246733 669 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:58.246990 670 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5698f79a071484096aaec5716230ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5698f79a071484096aaec5716230ab3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42865 } } }
I20260807 22:42:58.247080 670 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:42:58.249189 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.236005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.26.189.254:42865)
0807 22:42:58.236450 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:58.236466 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:58.236500 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:42:58.237166 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:58.237184 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:58.237577 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:58.237908 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:58.238731 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.239718 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.240559 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.241039 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.241189 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.243693 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:58.243698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:58.243710 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:58.243735 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:58.245652 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.246151 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.246157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.246162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.246330 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.246635 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.246639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.246640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.247154 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:58.247351 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:58.247632 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 22:42:58.248094 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":140,"threads_started":1}
W20260807 22:42:58.251307 697 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:42:58.251430 697 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:42:58.251524 674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:42:58.252417 674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:42:58.257612 674 catalog_manager.cc:1347] Generated new cluster ID: 370816eb8d0444dab525c910d869cd78
I20260807 22:42:58.257663 674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:42:58.266551 674 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:42:58.267364 674 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:42:58.271996 674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5698f79a071484096aaec5716230ab3: Generated new TSK 0
I20260807 22:42:58.272526 674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:42:58.303356 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42865
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:58.410948 705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:58.411177 705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:58.411221 705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:58.415169 705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:58.415217 705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:58.415287 705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:42:58.420312 705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:58.421243 705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:58.422200 705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:58.431137 713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.431160 710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.431325 705 server_base.cc:1034] running on GCE node
W20260807 22:42:58.431149 711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.431804 705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:58.432390 705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:58.433573 705 hybrid_clock.cc:648] HybridClock initialized: now 1786142578433541 us; error 41 us; skew 500 ppm
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:58.436305 705 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:42:58.437136 705 webserver.cc:460] Webserver started at http://127.26.189.193:39357/ using document root <none> and password file <none>
I20260807 22:42:58.437649 705 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:58.437714 705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:58.437911 705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:58.439917 705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "5ebcc476fa1e4c449954d1e9924ba052"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "e3d6321d821c90718baf693314ae440d"
server_key_iv: "9934fcc08ec8259670594152312a8ef6"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.440519 705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "5ebcc476fa1e4c449954d1e9924ba052"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "e3d6321d821c90718baf693314ae440d"
server_key_iv: "9934fcc08ec8259670594152312a8ef6"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.443997 705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 22:42:58.446352 723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.447572 705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 22:42:58.447676 705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "5ebcc476fa1e4c449954d1e9924ba052"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "e3d6321d821c90718baf693314ae440d"
server_key_iv: "9934fcc08ec8259670594152312a8ef6"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.447765 705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:58.487046 705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:58.490201 705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:58.490399 705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:58.490988 705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:58.491807 705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:58.491856 705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.491909 705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:58.491933 705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.500061 705 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:41293
I20260807 22:42:58.500101 891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:41293 every 8 connection(s)
I20260807 22:42:58.500917 705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:42:58.509425 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 705
I20260807 22:42:58.509560 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:42:58.509850 27383 external_mini_cluster.cc:1428] Setting key c9fc1837a836ba5ba18543193e846e27
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:58.513902 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.502648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:35439 (local address 127.26.189.254:42865)
0807 22:42:58.502768 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:58.502772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:58.503543 (+ 771us) server_negotiation.cc:407] Connection header received
0807 22:42:58.504994 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:58.504998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:58.505058 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:58.505142 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:58.506334 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.506871 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.508009 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.508252 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.508343 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.510722 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:58.510726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:58.510728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:58.510753 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:58.512031 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.512495 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.512498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.512499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.512574 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.512879 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.512882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.512884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.513058 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:58.513146 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:58.513621 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 22:42:58.513742 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:42:58.514741 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.502968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42865 (local address 127.26.189.193:35439)
0807 22:42:58.503396 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:58.503439 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:58.504794 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:58.505327 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:58.505334 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:58.505688 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:58.506152 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:58.506172 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.506994 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.506997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:58.507890 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:58.507900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.508392 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.508397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:58.508558 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.509068 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:58.509083 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:58.510538 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:58.512140 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:58.512146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:58.512161 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:58.512402 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:58.512656 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:58.512659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:58.512661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:58.512794 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:58.513166 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:58.513172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:58.513400 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 22:42:58.513557 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:42:58.513872 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260807 22:42:58.515877 894 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42865
I20260807 22:42:58.516140 894 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:58.516738 894 heartbeater.cc:511] Master 127.26.189.254:42865 requested a full tablet report, sending...
I20260807 22:42:58.518477 604 ts_manager.cc:194] Registered new tserver with Master: 5ebcc476fa1e4c449954d1e9924ba052 (127.26.189.193:41293)
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:42:58.520035 604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:35439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:42:58.536765 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42865
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:58.645171 908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:58.645416 908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:58.645466 908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:58.649582 908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:58.649636 908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:58.649689 908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:42:58.654704 908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:58.655578 908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:58.656469 908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:58.665374 914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.665570 908 server_base.cc:1034] running on GCE node
W20260807 22:42:58.665374 915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.665392 919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.665994 908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:58.666630 908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:58.667783 908 hybrid_clock.cc:648] HybridClock initialized: now 1786142578667759 us; error 36 us; skew 500 ppm
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:58.670368 908 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:42:58.671135 908 webserver.cc:460] Webserver started at http://127.26.189.194:41673/ using document root <none> and password file <none>
I20260807 22:42:58.671633 908 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:58.671696 908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:58.671873 908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:58.673799 908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "3d25fe7d03ed4bd99b9d8842f43784f3"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "44c4d533ddf1d6f5e7e44b427237b522"
server_key_iv: "beaed6b82e0bd4e2928cd923b92d4ebc"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.674371 908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "3d25fe7d03ed4bd99b9d8842f43784f3"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "44c4d533ddf1d6f5e7e44b427237b522"
server_key_iv: "beaed6b82e0bd4e2928cd923b92d4ebc"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.677750 908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:42:58.679960 926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.681069 908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:42:58.681172 908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "3d25fe7d03ed4bd99b9d8842f43784f3"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "44c4d533ddf1d6f5e7e44b427237b522"
server_key_iv: "beaed6b82e0bd4e2928cd923b92d4ebc"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.681258 908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:58.690857 908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:58.693331 908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:58.693492 908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:58.694023 908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:58.694878 908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:58.694934 908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.694988 908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:58.695015 908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.703229 908 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:40757
I20260807 22:42:58.703258 1118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:40757 every 8 connection(s)
I20260807 22:42:58.704085 908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:42:58.712880 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 908
I20260807 22:42:58.713007 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:42:58.713303 27383 external_mini_cluster.cc:1428] Setting key 6eeeff19f7dbfcdfcdce6168581d9f08
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:58.717481 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.705872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53977 (local address 127.26.189.254:42865)
0807 22:42:58.706013 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:58.706018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:58.706879 (+ 861us) server_negotiation.cc:407] Connection header received
0807 22:42:58.708240 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:58.708244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:58.708303 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:58.708380 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:58.709568 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.710107 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.711270 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.711496 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.711588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.714069 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:58.714074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:58.714077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:58.714112 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:58.715532 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.716005 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.716008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.716010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.716098 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.716391 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.716394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.716396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.716601 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:58.716694 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:58.717264 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 22:42:58.717362 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:42:58.718567 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.706255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42865 (local address 127.26.189.194:53977)
0807 22:42:58.706730 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:58.706774 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:58.708024 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:58.708562 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:58.708569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:58.708926 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:58.709412 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:58.709427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.710230 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.710232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:58.711143 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:58.711154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.711651 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.711656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:58.711820 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.712341 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:58.712357 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:58.713915 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:58.715648 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:58.715653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:58.715665 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:58.715918 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:58.716189 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:58.716191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:58.716192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:58.716320 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:58.716683 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:58.716688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:58.717001 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 22:42:58.717213 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 22:42:58.717551 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":118,"threads_started":1}
I20260807 22:42:58.719735 1119 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42865
I20260807 22:42:58.719977 1119 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:58.720496 1119 heartbeater.cc:511] Master 127.26.189.254:42865 requested a full tablet report, sending...
I20260807 22:42:58.721508 604 ts_manager.cc:194] Registered new tserver with Master: 3d25fe7d03ed4bd99b9d8842f43784f3 (127.26.189.194:40757)
I20260807 22:42:58.722044 604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:53977
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:42:58.740738 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42865
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:42:58.854236 1127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:42:58.854547 1127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:42:58.854614 1127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:42:58.858629 1127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:42:58.858690 1127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:42:58.858763 1127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:42:58.864118 1127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:42:58.865151 1127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:42:58.866180 1127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:42:58.875573 1139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:42:58.875594 1138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.875677 1127 server_base.cc:1034] running on GCE node
W20260807 22:42:58.875763 1141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:42:58.876209 1127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:42:58.876804 1127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:42:58.877955 1127 hybrid_clock.cc:648] HybridClock initialized: now 1786142578877939 us; error 28 us; skew 500 ppm
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:42:58.880586 1127 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:42:58.881462 1127 webserver.cc:460] Webserver started at http://127.26.189.195:40445/ using document root <none> and password file <none>
I20260807 22:42:58.881973 1127 fs_manager.cc:359] Metadata directory not provided
I20260807 22:42:58.882038 1127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:42:58.882239 1127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:42:58.884269 1127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "7bce013ade4d419b9fcd9411de6e2a4d"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "d743916d48fe0d7a5c96cfb369760792"
server_key_iv: "edb56e207f192c0d7d22d291d316aead"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.884912 1127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "7bce013ade4d419b9fcd9411de6e2a4d"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "d743916d48fe0d7a5c96cfb369760792"
server_key_iv: "edb56e207f192c0d7d22d291d316aead"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.888475 1127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 22:42:58.890730 1150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.891878 1127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:42:58.891985 1127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "7bce013ade4d419b9fcd9411de6e2a4d"
format_stamp: "Formatted at 2026-08-07 22:42:58 on dist-test-slave-4tdj"
server_key: "d743916d48fe0d7a5c96cfb369760792"
server_key_iv: "edb56e207f192c0d7d22d291d316aead"
server_key_version: "encryptionkey@0"
I20260807 22:42:58.892074 1127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:42:58.930658 1127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:42:58.933912 1127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:42:58.934100 1127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:42:58.934741 1127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:42:58.935608 1127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:42:58.935662 1127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.935717 1127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:42:58.935745 1127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:42:58.945750 1127 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:46561
I20260807 22:42:58.945796 1341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:46561 every 8 connection(s)
I20260807 22:42:58.946771 1127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:42:58.951992 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 1127
I20260807 22:42:58.952117 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:42:58.952442 27383 external_mini_cluster.cc:1428] Setting key fd69bb4762d4275076bce599435c2db8
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:58.960021 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.948554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:37163 (local address 127.26.189.254:42865)
0807 22:42:58.948673 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:58.948677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:58.949553 (+ 876us) server_negotiation.cc:407] Connection header received
0807 22:42:58.950975 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:58.950979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:58.951036 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:58.951111 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:58.952446 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.953047 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.954647 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.954862 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.954940 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.957234 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:58.957237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:58.957239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:58.957264 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:58.958401 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.958850 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.958853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.958854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.958922 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.959162 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.959164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.959165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.959324 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:58.959394 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:58.959816 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 22:42:58.959908 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:42:58.960755 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.948931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42865 (local address 127.26.189.195:37163)
0807 22:42:58.949371 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 22:42:58.949413 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:42:58.950757 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:42:58.951287 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:42:58.951293 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:42:58.951626 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:42:58.952271 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:58.952292 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.953170 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.953172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:58.954449 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:42:58.954462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.955016 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.955019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:42:58.955174 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.955650 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:42:58.955666 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:42:58.957072 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:42:58.958506 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:58.958511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:58.958521 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:58.958777 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:58.958983 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:42:58.958985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:42:58.958986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:42:58.959103 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:42:58.959392 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:42:58.959396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:42:58.959600 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 22:42:58.959757 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:42:58.959985 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":120,"threads_started":1}
I20260807 22:42:58.961772 1342 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42865
I20260807 22:42:58.961985 1342 heartbeater.cc:464] Registering TS with master...
I20260807 22:42:58.962458 1342 heartbeater.cc:511] Master 127.26.189.254:42865 requested a full tablet report, sending...
I20260807 22:42:58.963402 604 ts_manager.cc:194] Registered new tserver with Master: 7bce013ade4d419b9fcd9411de6e2a4d (127.26.189.195:46561)
I20260807 22:42:58.963928 604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:37163
I20260807 22:42:58.966063 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:42:58.991439 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.982621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.26.189.193:41293)
0807 22:42:58.982906 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:58.982920 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:58.982943 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:42:58.982993 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:58.982998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:58.983179 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:58.983298 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:58.983888 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.984734 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.985638 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.985997 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.986080 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.988219 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:58.988228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:58.988241 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:58.988266 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:58.989743 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.990062 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.990066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.990068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.990144 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:58.990369 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:58.990371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:58.990372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:58.990614 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:58.990780 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:58.991113 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 22:42:58.991246 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
Aug 07 22:42:58 dist-test-slave-4tdj krb5kdc[540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:42:59.001430 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:42:58.994158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.26.189.254:42865)
0807 22:42:58.994335 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 22:42:58.994340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:42:58.994356 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:42:58.994467 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:42:58.994469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:42:58.994510 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:42:58.994571 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:42:58.995128 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.995680 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.996412 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:42:58.996671 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:42:58.996746 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:42:58.998971 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:42:58.998974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:42:58.998976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:42:58.998999 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:42:59.000143 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.000510 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.000512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.000513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.000578 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:42:59.000749 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:42:59.000751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:42:59.000752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:42:59.000941 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:42:59.001035 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:42:59.001189 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 22:42:59.001278 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 22:42:59.001843 604 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35428
I20260807 22:42:59.002430 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 705
I20260807 22:42:59.027655 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 908
I20260807 22:42:59.042866 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 1127
I20260807 22:42:59.056545 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 561
2026-08-07T22:42:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2242 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:42:59 dist-test-slave-4tdj krb5kdc[1359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:42:59 dist-test-slave-4tdj krb5kdc[1359](info): set up 2 sockets
Aug 07 22:42:59 dist-test-slave-4tdj krb5kdc[1359](info): commencing operation
krb5kdc: starting...
W20260807 22:43:00.192160 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:43:00.310847 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:45101
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:45101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:00.420722 1380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:00.420979 1380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:00.421022 1380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:00.424995 1380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:00.425055 1380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:00.425074 1380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:00.425109 1380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:00.425130 1380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:00.430335 1380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:45101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:45101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:00.431362 1380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:00.432375 1380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:00.440531 1390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.440546 1393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.440673 1380 server_base.cc:1034] running on GCE node
W20260807 22:43:00.440526 1389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.441349 1380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:00.442457 1380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:00.443605 1380 hybrid_clock.cc:648] HybridClock initialized: now 1786142580443584 us; error 36 us; skew 500 ppm
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:00.449101 1380 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:43:00.449812 1380 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:00.449877 1380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:00.450075 1380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:00.452144 1380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "12bd93e9d5844eb698feefb31eed880a"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "63f655019516d63dd3080332318e88b1"
server_key_iv: "87a54b23000b9e8ea9cbe2d019320237"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.452704 1380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "12bd93e9d5844eb698feefb31eed880a"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "63f655019516d63dd3080332318e88b1"
server_key_iv: "87a54b23000b9e8ea9cbe2d019320237"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.456247 1380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:00.458501 1407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.459692 1380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:00.459821 1380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "12bd93e9d5844eb698feefb31eed880a"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "63f655019516d63dd3080332318e88b1"
server_key_iv: "87a54b23000b9e8ea9cbe2d019320237"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.459913 1380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:00.473577 1380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:00.476099 1380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:00.476255 1380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:00.482846 1380 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:45101
I20260807 22:43:00.483065 1491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:45101 every 8 connection(s)
I20260807 22:43:00.483457 1380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:00.486020 1493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:00.487025 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 1380
I20260807 22:43:00.487155 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:00.487485 27383 external_mini_cluster.cc:1428] Setting key 49dc7f2bbf3cfc17f92229181ba4a29b
I20260807 22:43:00.492044 1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a: Bootstrap starting.
I20260807 22:43:00.494645 1493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:00.495529 1493 log.cc:824] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:00.497614 1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a: No bootstrap required, opened a new log
I20260807 22:43:00.501008 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.488729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.26.189.254:45101)
0807 22:43:00.489127 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.489142 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.489178 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:43:00.489808 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.489824 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.490146 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.490441 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.491322 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.492273 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.493383 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.493820 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.493924 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.496644 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.496649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.496661 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.496684 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.498376 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.498827 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.498831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.498835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.498984 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.499244 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.499246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.499247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.499628 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.499776 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.500043 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.500318 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260807 22:43:00.501350 1493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bd93e9d5844eb698feefb31eed880a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45101 } }
I20260807 22:43:00.501556 1493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:00.501611 1493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12bd93e9d5844eb698feefb31eed880a, State: Initialized, Role: FOLLOWER
I20260807 22:43:00.501993 1493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bd93e9d5844eb698feefb31eed880a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45101 } }
I20260807 22:43:00.502136 1493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:00.502206 1493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:00.502318 1493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:00.503312 1493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bd93e9d5844eb698feefb31eed880a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45101 } }
I20260807 22:43:00.503615 1493 leader_election.cc:302] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12bd93e9d5844eb698feefb31eed880a; no voters:
I20260807 22:43:00.503868 1493 leader_election.cc:288] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:00.503979 1500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:00.504211 1500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [term 1 LEADER]: Becoming Leader. State: Replica: 12bd93e9d5844eb698feefb31eed880a, State: Running, Role: LEADER
I20260807 22:43:00.504484 1500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bd93e9d5844eb698feefb31eed880a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45101 } }
I20260807 22:43:00.504935 1493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:00.505909 1502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12bd93e9d5844eb698feefb31eed880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bd93e9d5844eb698feefb31eed880a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45101 } } }
I20260807 22:43:00.506050 1502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:00.506378 1503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12bd93e9d5844eb698feefb31eed880a. Latest consensus state: current_term: 1 leader_uuid: "12bd93e9d5844eb698feefb31eed880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bd93e9d5844eb698feefb31eed880a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45101 } } }
I20260807 22:43:00.506469 1503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:00.506549 1523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:00.509135 1523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:00.514236 1523 catalog_manager.cc:1347] Generated new cluster ID: 07724a88b1444c1d8de6ce548758e1d0
I20260807 22:43:00.514334 1523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:00.530246 1523 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:00.531571 1523 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:00.540156 1523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a: Generated new TSK 0
I20260807 22:43:00.540920 1523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:43:00.573118 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:45101
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:00.681236 1541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:00.681495 1541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:00.681545 1541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:00.685552 1541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:00.685613 1541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:00.685688 1541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:00.691082 1541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:45101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:00.692086 1541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:00.693076 1541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:00.701946 1553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.701997 1547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.701969 1549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.702042 1541 server_base.cc:1034] running on GCE node
I20260807 22:43:00.702611 1541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:00.703224 1541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:00.704383 1541 hybrid_clock.cc:648] HybridClock initialized: now 1786142580704352 us; error 49 us; skew 500 ppm
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:00.707149 1541 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:43:00.707810 1541 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:00.707878 1541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:00.708077 1541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:00.710047 1541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "b86188be7705481e8a8529acf1630fe7"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "04c021fb0b0db3c868bc241b9ef9fd06"
server_key_iv: "e5584ae24ac0e06b7afc54e54d2a77cd"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.710609 1541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "b86188be7705481e8a8529acf1630fe7"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "04c021fb0b0db3c868bc241b9ef9fd06"
server_key_iv: "e5584ae24ac0e06b7afc54e54d2a77cd"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.713980 1541 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:00.716252 1562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.717378 1541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:00.717480 1541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "b86188be7705481e8a8529acf1630fe7"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "04c021fb0b0db3c868bc241b9ef9fd06"
server_key_iv: "e5584ae24ac0e06b7afc54e54d2a77cd"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.717569 1541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:00.726480 1541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:00.729378 1541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:00.729543 1541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:00.730093 1541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:00.730930 1541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:00.730978 1541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.731030 1541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:00.731058 1541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.739248 1541 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:35353
I20260807 22:43:00.739279 1740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:35353 every 8 connection(s)
I20260807 22:43:00.739797 1541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:00.749162 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 1541
I20260807 22:43:00.749296 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:00.749624 27383 external_mini_cluster.cc:1428] Setting key 2eea0bd1212799e242960e31b4d3d72c
I20260807 22:43:00.752421 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.741390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:33039 (local address 127.26.189.254:45101)
0807 22:43:00.741530 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.741535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.742397 (+ 862us) server_negotiation.cc:407] Connection header received
0807 22:43:00.743685 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.743688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.743733 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.743810 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.744897 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.745433 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.746538 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.746771 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.746844 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.749052 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.749057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.749060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.749099 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.750483 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.751020 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.751023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.751025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.751102 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.751412 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.751416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.751417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.751591 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.751678 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.752143 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.752266 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:43:00.753269 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.741750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45101 (local address 127.26.189.193:33039)
0807 22:43:00.742208 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:00.742258 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:00.743508 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:00.743959 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:00.743965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:00.744295 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:00.744770 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.744784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.745530 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.745533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.746451 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.746461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.746881 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.746884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.747018 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.747520 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:00.747536 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:00.748889 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:00.750585 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.750591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.750612 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.750921 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.751190 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.751193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.751195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.751327 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.751691 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:00.751697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:00.751920 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 22:43:00.752077 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:43:00.752377 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":115,"threads_started":1}
I20260807 22:43:00.754410 1741 heartbeater.cc:347] Connected to a master server at 127.26.189.254:45101
I20260807 22:43:00.754624 1741 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:00.755028 1741 heartbeater.cc:511] Master 127.26.189.254:45101 requested a full tablet report, sending...
I20260807 22:43:00.756563 1430 ts_manager.cc:194] Registered new tserver with Master: b86188be7705481e8a8529acf1630fe7 (127.26.189.193:35353)
I20260807 22:43:00.758116 1430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:33039
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:43:00.777122 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:45101
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:00.886250 1750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:00.886530 1750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:00.886579 1750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:00.890626 1750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:00.890679 1750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:00.890748 1750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:00.896543 1750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:45101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:00.897540 1750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:00.898581 1750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:00.907752 1757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.907917 1750 server_base.cc:1034] running on GCE node
W20260807 22:43:00.907949 1759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:00.907729 1764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:00.908404 1750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:00.909024 1750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:00.910205 1750 hybrid_clock.cc:648] HybridClock initialized: now 1786142580910189 us; error 27 us; skew 500 ppm
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:00.912930 1750 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:43:00.913594 1750 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:00.913671 1750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:00.913869 1750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:00.915869 1750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "8b65c7290af046319ae1720bc9825459"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "279fef295937acc380380954bf9727b2"
server_key_iv: "237a6f2d344eddfc1276e8222a86afb1"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.916411 1750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "8b65c7290af046319ae1720bc9825459"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "279fef295937acc380380954bf9727b2"
server_key_iv: "237a6f2d344eddfc1276e8222a86afb1"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.919885 1750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:00.922281 1774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.923496 1750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:00.923600 1750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "8b65c7290af046319ae1720bc9825459"
format_stamp: "Formatted at 2026-08-07 22:43:00 on dist-test-slave-4tdj"
server_key: "279fef295937acc380380954bf9727b2"
server_key_iv: "237a6f2d344eddfc1276e8222a86afb1"
server_key_version: "encryptionkey@0"
I20260807 22:43:00.923703 1750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:00.936092 1750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:00.939206 1750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:00.939386 1750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:00.940467 1750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:00.941243 1750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:00.941290 1750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.941329 1750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:00.941349 1750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:00.949360 1750 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:44327
I20260807 22:43:00.949383 1959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:44327 every 8 connection(s)
I20260807 22:43:00.949903 1750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:00.953279 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 1750
I20260807 22:43:00.953408 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:00.953739 27383 external_mini_cluster.cc:1428] Setting key 0db5c503731d86e9aa12237e95bd0d98
Aug 07 22:43:00 dist-test-slave-4tdj krb5kdc[1359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142580, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:00.964301 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.951636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:39013 (local address 127.26.189.254:45101)
0807 22:43:00.951806 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:00.951811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:00.952592 (+ 781us) server_negotiation.cc:407] Connection header received
0807 22:43:00.954083 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:00.954087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:00.954157 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:00.954246 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:00.955679 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.956193 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.957639 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.957974 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.958083 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.960718 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:00.960721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:00.960723 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:00.960749 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:00.962008 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.962617 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.962620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.962621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.962700 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:00.963123 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:00.963126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:00.963127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:00.963319 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:00.963396 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:00.964035 (+ 639us) server_negotiation.cc:299] Negotiation successful
0807 22:43:00.964166 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:43:00.965265 1965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:00.952008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45101 (local address 127.26.189.194:39013)
0807 22:43:00.952445 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:00.952486 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:00.953869 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:00.954469 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:00.954478 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:00.954892 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:00.955526 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.955546 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.956315 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.956319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.957523 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:00.957533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:00.958138 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:00.958142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:00.958288 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:00.958882 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:00.958901 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:00.960536 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:00.962127 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.962134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.962151 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.962511 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.962788 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:00.962791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:00.962793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:00.962974 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:00.963432 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:00.963439 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:00.963773 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 22:43:00.963991 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 22:43:00.964407 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260807 22:43:00.966353 1962 heartbeater.cc:347] Connected to a master server at 127.26.189.254:45101
I20260807 22:43:00.966614 1962 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:00.967065 1962 heartbeater.cc:511] Master 127.26.189.254:45101 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
I20260807 22:43:00.968058 1430 ts_manager.cc:194] Registered new tserver with Master: 8b65c7290af046319ae1720bc9825459 (127.26.189.194:44327)
I20260807 22:43:00.968618 1430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:39013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:43:00.980970 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:45101
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:01.091681 1968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:01.091914 1968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:01.091955 1968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:01.095785 1968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:01.095829 1968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:01.095883 1968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:01.100934 1968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:45101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:01.101854 1968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:01.102792 1968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:01.111706 1979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:01.111706 1975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.111923 1968 server_base.cc:1034] running on GCE node
W20260807 22:43:01.111940 1977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.112402 1968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:01.113030 1968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:01.114193 1968 hybrid_clock.cc:648] HybridClock initialized: now 1786142581114176 us; error 28 us; skew 500 ppm
Aug 07 22:43:01 dist-test-slave-4tdj krb5kdc[1359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142581, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:01.116873 1968 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:43:01.117579 1968 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:01.117650 1968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:01.117847 1968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:01.119905 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "66a7cbf14d78484bbab360292d35a9c7"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "79334db4ee58c67b806ef6089a953dc6"
server_key_iv: "fa9da239af7915ba1f367ae463468603"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.120438 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "66a7cbf14d78484bbab360292d35a9c7"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "79334db4ee58c67b806ef6089a953dc6"
server_key_iv: "fa9da239af7915ba1f367ae463468603"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.124027 1968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:01.126271 1987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.127525 1968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:01.127627 1968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "66a7cbf14d78484bbab360292d35a9c7"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "79334db4ee58c67b806ef6089a953dc6"
server_key_iv: "fa9da239af7915ba1f367ae463468603"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.127714 1968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:01.140399 1968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:01.142932 1968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:01.143092 1968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:01.143625 1968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:01.144392 1968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:01.144438 1968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.144497 1968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:01.144526 1968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.152904 1968 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:40615
I20260807 22:43:01.152922 2167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:40615 every 8 connection(s)
I20260807 22:43:01.153431 1968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:01.156913 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 1968
I20260807 22:43:01.157014 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:01.157284 27383 external_mini_cluster.cc:1428] Setting key 5319679ec472ec51aa44dc22b0bf17ec
Aug 07 22:43:01 dist-test-slave-4tdj krb5kdc[1359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142581, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:01.166452 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.155058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:55019 (local address 127.26.189.254:45101)
0807 22:43:01.155229 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.155234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.156020 (+ 786us) server_negotiation.cc:407] Connection header received
0807 22:43:01.157352 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.157355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.157412 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.157501 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.158823 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.159470 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.160554 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.160771 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.160842 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.163186 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.163189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.163191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:01.163216 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.164628 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.165063 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.165065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.165067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.165145 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.165447 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.165451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.165452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.165690 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.165775 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.166181 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.166315 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:43:01.167249 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.155412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45101 (local address 127.26.189.195:55019)
0807 22:43:01.155872 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.155913 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.157150 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.157711 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.157720 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.158094 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:01.158698 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.158713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.159584 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.159587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.160446 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.160456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.160902 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.160905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.161053 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.161597 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:01.161614 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:01.163047 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:01.164728 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:01.164732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:01.164744 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:01.164984 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:01.165217 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:01.165219 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:01.165221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:01.165341 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:01.165779 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:01.165783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:01.165999 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.166146 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.166449 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
I20260807 22:43:01.168390 2169 heartbeater.cc:347] Connected to a master server at 127.26.189.254:45101
I20260807 22:43:01.168592 2169 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:01.168987 2169 heartbeater.cc:511] Master 127.26.189.254:45101 requested a full tablet report, sending...
I20260807 22:43:01.169950 1430 ts_manager.cc:194] Registered new tserver with Master: 66a7cbf14d78484bbab360292d35a9c7 (127.26.189.195:40615)
I20260807 22:43:01.170581 1430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:55019
I20260807 22:43:01.171039 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:43:01.179277 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.172879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.26.189.254:45101)
0807 22:43:01.172992 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.172995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.173155 (+ 160us) server_negotiation.cc:407] Connection header received
0807 22:43:01.173244 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.173246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.173281 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.173342 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.173915 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.174450 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.175214 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.175419 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.175512 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.176642 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.176645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.176647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:01.176671 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.177837 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.178208 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.178210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.178211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.178381 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:01.178595 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:01.178597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:01.178598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:01.178807 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.178919 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.179078 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.179167 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:43:01.181685 1430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:01.184643 1430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:01.198834 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.192677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44327 (local address 127.0.0.1:34796)
0807 22:43:01.193111 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.193156 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.193283 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.193856 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.193863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.193886 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.194028 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.194035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.195425 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.195428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.196368 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.196377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.198316 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.198321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.198439 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.198489 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.198570 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.198687 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":67,"threads_started":1}
I20260807 22:43:01.199716 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.192813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.26.189.194:44327)
0807 22:43:01.193244 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.193257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.193281 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:43:01.193429 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.193435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.193667 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.194182 (+ 515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.194376 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.195314 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.196480 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.198151 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.199280 (+ 1129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.199413 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.199467 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.199564 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":124,"threads_started":1}
I20260807 22:43:01.200634 2191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.192800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:40615 (local address 127.0.0.1:40570)
0807 22:43:01.193591 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.193604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.193687 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.194838 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.194841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.194858 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.194932 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.194938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.196251 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.196255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.198243 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.198256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.200255 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.200258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.200345 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.200366 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.200440 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.200506 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":153,"thread_start_us":93,"threads_started":1}
I20260807 22:43:01.201172 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.193410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.26.189.195:40615)
0807 22:43:01.194244 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.194255 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.194276 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:43:01.194367 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.194371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.194549 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.194690 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.195051 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.196115 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.198477 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.199947 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.200062 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.200191 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.200538 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.200670 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":129,"threads_started":1}
I20260807 22:43:01.201256 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.193340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:35353 (local address 127.0.0.1:35722)
0807 22:43:01.193893 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.193905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.193979 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.194629 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.194632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.194648 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.194745 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.194751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.196090 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.196093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.197400 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.197412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.200900 (+ 3488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.200904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.200989 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.201012 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.201059 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.201132 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":50,"thread_start_us":95,"threads_started":1}
I20260807 22:43:01.202246 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.193760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.26.189.193:35353)
0807 22:43:01.194185 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.194198 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.194221 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:43:01.194319 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.194325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.194501 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.194633 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.194836 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.195933 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.197561 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.198364 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.201810 (+ 3446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.201938 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.201988 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.202080 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":105,"threads_started":1}
I20260807 22:43:01.203945 1844 tablet_service.cc:1467] Processing CreateTablet for tablet 72509623913f4b57b90d17b9b8e16bc2 (DEFAULT_TABLE table=test-table [id=231c0e0b48c74304bb5c3beda64d9b41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.204291 2063 tablet_service.cc:1467] Processing CreateTablet for tablet 72509623913f4b57b90d17b9b8e16bc2 (DEFAULT_TABLE table=test-table [id=231c0e0b48c74304bb5c3beda64d9b41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.205133 1844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72509623913f4b57b90d17b9b8e16bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.205591 2063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72509623913f4b57b90d17b9b8e16bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.205492 1642 tablet_service.cc:1467] Processing CreateTablet for tablet 72509623913f4b57b90d17b9b8e16bc2 (DEFAULT_TABLE table=test-table [id=231c0e0b48c74304bb5c3beda64d9b41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:01.206517 1642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72509623913f4b57b90d17b9b8e16bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.211687 2194 tablet_bootstrap.cc:492] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Bootstrap starting.
I20260807 22:43:01.213660 2197 tablet_bootstrap.cc:492] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Bootstrap starting.
I20260807 22:43:01.213956 2194 tablet_bootstrap.cc:654] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.214494 2196 tablet_bootstrap.cc:492] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: Bootstrap starting.
I20260807 22:43:01.215451 2194 log.cc:824] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.216612 2196 tablet_bootstrap.cc:654] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.216696 2197 tablet_bootstrap.cc:654] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.217708 2194 tablet_bootstrap.cc:492] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: No bootstrap required, opened a new log
I20260807 22:43:01.217841 2196 log.cc:824] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.217904 2194 ts_tablet_manager.cc:1397] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:43:01.217912 2197 log.cc:824] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.219816 2196 tablet_bootstrap.cc:492] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: No bootstrap required, opened a new log
I20260807 22:43:01.219820 2197 tablet_bootstrap.cc:492] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: No bootstrap required, opened a new log
I20260807 22:43:01.220002 2196 ts_tablet_manager.cc:1397] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:43:01.220001 2197 ts_tablet_manager.cc:1397] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 22:43:01.221858 2194 raft_consensus.cc:348] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.222164 2194 raft_consensus.cc:374] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.222221 2194 raft_consensus.cc:729] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a7cbf14d78484bbab360292d35a9c7, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.222708 2194 consensus_queue.cc:260] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.223385 2169 heartbeater.cc:503] Master 127.26.189.254:45101 was elected leader, sending a full tablet report...
I20260807 22:43:01.223713 2194 ts_tablet_manager.cc:1428] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 22:43:01.223624 2196 raft_consensus.cc:348] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.223853 2196 raft_consensus.cc:374] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.223898 2196 raft_consensus.cc:729] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86188be7705481e8a8529acf1630fe7, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.223937 2197 raft_consensus.cc:348] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.224200 2197 raft_consensus.cc:374] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.224249 2197 raft_consensus.cc:729] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b65c7290af046319ae1720bc9825459, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.224284 2196 consensus_queue.cc:260] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.224687 2197 consensus_queue.cc:260] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.225248 2196 ts_tablet_manager.cc:1428] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:43:01.225602 2197 ts_tablet_manager.cc:1428] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:43:01.225610 1741 heartbeater.cc:503] Master 127.26.189.254:45101 was elected leader, sending a full tablet report...
I20260807 22:43:01.225943 1962 heartbeater.cc:503] Master 127.26.189.254:45101 was elected leader, sending a full tablet report...
W20260807 22:43:01.240949 1742 tablet.cc:2369] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:01.292075 2206 raft_consensus.cc:482] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:01.292294 2206 raft_consensus.cc:504] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.293417 2206 leader_election.cc:288] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b65c7290af046319ae1720bc9825459 (127.26.189.194:44327), b86188be7705481e8a8529acf1630fe7 (127.26.189.193:35353)
I20260807 22:43:01.297263 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.293651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44327 (local address 127.26.189.195:34805)
0807 22:43:01.293779 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.293794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.293885 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.294145 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.294149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.294171 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.294322 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.294329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.295180 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.295183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.296134 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.296144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.296863 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.296867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.296974 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.296999 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.297070 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.297140 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":22}
I20260807 22:43:01.297294 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.293745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:34805 (local address 127.26.189.194:44327)
0807 22:43:01.293857 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.293863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.293880 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:01.293980 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.293983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.294031 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.294126 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.294425 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.295069 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.296257 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.296751 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.296841 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.296883 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.297088 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.297169 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:43:01.297477 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.293919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:35353 (local address 127.26.189.195:45519)
0807 22:43:01.294316 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.294329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.294424 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.294639 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.294642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.294658 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:01.294764 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.294770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.295555 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.295558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.296393 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:01.296403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.297082 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.297086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:01.297187 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.297210 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.297284 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.297349 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":121,"threads_started":1}
I20260807 22:43:01.297873 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.294049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:45519 (local address 127.26.189.193:35353)
0807 22:43:01.294274 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.294279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.294405 (+ 126us) server_negotiation.cc:407] Connection header received
0807 22:43:01.294508 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.294510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.294553 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.294606 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:01.294849 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.295448 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.296528 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.296962 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.297093 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.297138 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.297309 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.297782 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260807 22:43:01.297904 1877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72509623913f4b57b90d17b9b8e16bc2" candidate_uuid: "66a7cbf14d78484bbab360292d35a9c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b65c7290af046319ae1720bc9825459" is_pre_election: true
I20260807 22:43:01.298226 1877 raft_consensus.cc:2454] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a7cbf14d78484bbab360292d35a9c7 in term 0.
I20260807 22:43:01.298426 1679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72509623913f4b57b90d17b9b8e16bc2" candidate_uuid: "66a7cbf14d78484bbab360292d35a9c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86188be7705481e8a8529acf1630fe7" is_pre_election: true
I20260807 22:43:01.298717 1679 raft_consensus.cc:2454] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a7cbf14d78484bbab360292d35a9c7 in term 0.
I20260807 22:43:01.298772 1989 leader_election.cc:302] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a7cbf14d78484bbab360292d35a9c7, 8b65c7290af046319ae1720bc9825459; no voters:
I20260807 22:43:01.299037 2206 raft_consensus.cc:2781] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:01.299145 2206 raft_consensus.cc:482] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:01.299202 2206 raft_consensus.cc:3037] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.300223 2206 raft_consensus.cc:504] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.300505 2206 leader_election.cc:288] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [CANDIDATE]: Term 1 election: Requested vote from peers 8b65c7290af046319ae1720bc9825459 (127.26.189.194:44327), b86188be7705481e8a8529acf1630fe7 (127.26.189.193:35353)
I20260807 22:43:01.300918 1679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72509623913f4b57b90d17b9b8e16bc2" candidate_uuid: "66a7cbf14d78484bbab360292d35a9c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b86188be7705481e8a8529acf1630fe7"
I20260807 22:43:01.300918 1877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72509623913f4b57b90d17b9b8e16bc2" candidate_uuid: "66a7cbf14d78484bbab360292d35a9c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b65c7290af046319ae1720bc9825459"
I20260807 22:43:01.301036 1877 raft_consensus.cc:3037] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.301044 1679 raft_consensus.cc:3037] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.302335 1679 raft_consensus.cc:2454] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a7cbf14d78484bbab360292d35a9c7 in term 1.
I20260807 22:43:01.302335 1877 raft_consensus.cc:2454] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a7cbf14d78484bbab360292d35a9c7 in term 1.
I20260807 22:43:01.302606 1989 leader_election.cc:302] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a7cbf14d78484bbab360292d35a9c7, 8b65c7290af046319ae1720bc9825459; no voters:
I20260807 22:43:01.302839 2206 raft_consensus.cc:2781] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:01.303043 2206 raft_consensus.cc:686] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 1 LEADER]: Becoming Leader. State: Replica: 66a7cbf14d78484bbab360292d35a9c7, State: Running, Role: LEADER
I20260807 22:43:01.303360 2206 consensus_queue.cc:237] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } }
I20260807 22:43:01.306478 1430 catalog_manager.cc:5495] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66a7cbf14d78484bbab360292d35a9c7 (127.26.189.195). New cstate: current_term: 1 leader_uuid: "66a7cbf14d78484bbab360292d35a9c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66a7cbf14d78484bbab360292d35a9c7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 40615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:01.323037 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.319600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.26.189.195:40615)
0807 22:43:01.319724 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.319729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.319743 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:43:01.319868 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.319870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.319921 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.320006 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:01.320255 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.320807 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.321629 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:01.321852 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:01.321946 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:01.322223 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:01.322658 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:01.322764 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.322832 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.322887 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:43:01.328641 1679 raft_consensus.cc:1261] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7 [term 1 FOLLOWER]: Refusing update from remote peer 66a7cbf14d78484bbab360292d35a9c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:01.328682 1877 raft_consensus.cc:1261] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 1 FOLLOWER]: Refusing update from remote peer 66a7cbf14d78484bbab360292d35a9c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:01.329269 2206 consensus_queue.cc:1035] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86188be7705481e8a8529acf1630fe7" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 35353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:01.329460 2213 consensus_queue.cc:1035] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b65c7290af046319ae1720bc9825459" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:01.335462 2217 mvcc.cc:205] Tried to move back new op lower bound from 7316040013115457536 to 7316040013021184000. Current Snapshot: MvccSnapshot[applied={T|T < 7316040013115457536}]
I20260807 22:43:01.337802 2218 mvcc.cc:205] Tried to move back new op lower bound from 7316040013115457536 to 7316040013021184000. Current Snapshot: MvccSnapshot[applied={T|T < 7316040013115457536}]
I20260807 22:43:01.343940 1430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:01.344137 1430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:01.346540 1430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12bd93e9d5844eb698feefb31eed880a: Sending DeleteTablet for 3 replicas of tablet 72509623913f4b57b90d17b9b8e16bc2
I20260807 22:43:01.347325 1844 tablet_service.cc:1514] Processing DeleteTablet for tablet 72509623913f4b57b90d17b9b8e16bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:01 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:34796
I20260807 22:43:01.347358 2063 tablet_service.cc:1514] Processing DeleteTablet for tablet 72509623913f4b57b90d17b9b8e16bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:01 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:40570
I20260807 22:43:01.347456 1642 tablet_service.cc:1514] Processing DeleteTablet for tablet 72509623913f4b57b90d17b9b8e16bc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:01 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:35722
I20260807 22:43:01.347877 2238 tablet_replica.cc:330] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: stopping tablet replica
I20260807 22:43:01.348101 2236 tablet_replica.cc:330] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: stopping tablet replica
I20260807 22:43:01.348194 2238 raft_consensus.cc:2229] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:01.348354 2237 tablet_replica.cc:330] T 72509623913f4b57b90d17b9b8e16bc2 P b86188be7705481e8a8529acf1630fe7: stopping tablet replica
I20260807 22:43:01.348429 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 1541
I20260807 22:43:01.348608 2238 raft_consensus.cc:2258] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:01.349035 2236 raft_consensus.cc:2229] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:01.351284 2236 raft_consensus.cc:2258] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:01.352088 2238 ts_tablet_manager.cc:1905] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:01.352833 2236 ts_tablet_manager.cc:1905] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:01.354956 2238 ts_tablet_manager.cc:1918] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:01.355060 2238 log.cc:1196] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/72509623913f4b57b90d17b9b8e16bc2
I20260807 22:43:01.355382 2238 ts_tablet_manager.cc:1939] T 72509623913f4b57b90d17b9b8e16bc2 P 66a7cbf14d78484bbab360292d35a9c7: Deleting consensus metadata
I20260807 22:43:01.355996 2236 ts_tablet_manager.cc:1918] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:01.356189 2236 log.cc:1196] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/72509623913f4b57b90d17b9b8e16bc2
I20260807 22:43:01.356631 2236 ts_tablet_manager.cc:1939] T 72509623913f4b57b90d17b9b8e16bc2 P 8b65c7290af046319ae1720bc9825459: Deleting consensus metadata
I20260807 22:43:01.356598 1415 catalog_manager.cc:4841] TS 66a7cbf14d78484bbab360292d35a9c7 (127.26.189.195:40615): tablet 72509623913f4b57b90d17b9b8e16bc2 (table test-table [id=231c0e0b48c74304bb5c3beda64d9b41]) successfully deleted
I20260807 22:43:01.357875 1413 catalog_manager.cc:4841] TS 8b65c7290af046319ae1720bc9825459 (127.26.189.194:44327): tablet 72509623913f4b57b90d17b9b8e16bc2 (table test-table [id=231c0e0b48c74304bb5c3beda64d9b41]) successfully deleted
W20260807 22:43:01.362829 1413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:35353 (error 108)
I20260807 22:43:01.362944 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 1750
I20260807 22:43:01.365633 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.365291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:35353 (local address 127.0.0.1:35736)
0807 22:43:01.365490 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.365545 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260807 22:43:01.366071 1413 catalog_manager.cc:4584] TS b86188be7705481e8a8529acf1630fe7 (127.26.189.193:35353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72509623913f4b57b90d17b9b8e16bc2: Network error: Client connection negotiation failed: client connection to 127.26.189.193:35353: connect: Connection refused (error 111)
I20260807 22:43:01.375795 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 1968
I20260807 22:43:01.387790 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 1380
2026-08-07T22:43:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2331 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T22:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:01Z Disabled control of system clock
I20260807 22:43:01.423076 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:41411
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:41411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:43:01.530820 2245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:01.531052 2245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:01.531086 2245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:01.534914 2245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:01.534956 2245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:01.534970 2245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:01.534984 2245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:01.535000 2245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:01.539686 2245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:41411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:41411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:01.540665 2245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:01.541940 2245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:01.549881 2254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:01.549887 2258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:01.549860 2256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.549934 2245 server_base.cc:1034] running on GCE node
I20260807 22:43:01.550631 2245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:01.551637 2245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:01.552791 2245 hybrid_clock.cc:648] HybridClock initialized: now 1786142581552773 us; error 30 us; skew 500 ppm
I20260807 22:43:01.554540 2245 webserver.cc:460] Webserver started at http://127.26.189.254:41337/ using document root <none> and password file <none>
I20260807 22:43:01.555066 2245 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:01.555125 2245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:01.555320 2245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:01.557355 2245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "5b96a0f742604944abe99cc353e8749a"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "f32ab1933985f98792b2e2de40dd563b"
server_key_iv: "fe267a88035d789df2973dd692d8cda1"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.557890 2245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "5b96a0f742604944abe99cc353e8749a"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "f32ab1933985f98792b2e2de40dd563b"
server_key_iv: "fe267a88035d789df2973dd692d8cda1"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.561367 2245 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:43:01.563746 2267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.564950 2245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:01.565065 2245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "5b96a0f742604944abe99cc353e8749a"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "f32ab1933985f98792b2e2de40dd563b"
server_key_iv: "fe267a88035d789df2973dd692d8cda1"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.565169 2245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:01.569507 2245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:01.570047 2245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:01.570200 2245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:01.576514 2245 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:41411
I20260807 22:43:01.576536 2357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:41411 every 8 connection(s)
I20260807 22:43:01.577351 2245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:01.578469 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 2245
I20260807 22:43:01.578572 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:01.578817 27383 external_mini_cluster.cc:1428] Setting key d9009bb913afd3adb898c8f46af77c11
I20260807 22:43:01.581341 2358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:01.584268 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.579990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.26.189.254:41411)
0807 22:43:01.580976 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.580990 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.581023 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:43:01.581685 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.581702 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.582044 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.582382 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.583140 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.583144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.583155 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:01.583166 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.583218 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:01.583232 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.583356 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.583425 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.583653 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":542,"threads_started":1}
I20260807 22:43:01.587155 2358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a: Bootstrap starting.
I20260807 22:43:01.589327 2358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:01.590024 2358 log.cc:824] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:01.591743 2358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a: No bootstrap required, opened a new log
I20260807 22:43:01.595095 2358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96a0f742604944abe99cc353e8749a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41411 } }
I20260807 22:43:01.595356 2358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:01.595405 2358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b96a0f742604944abe99cc353e8749a, State: Initialized, Role: FOLLOWER
I20260807 22:43:01.595757 2358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96a0f742604944abe99cc353e8749a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41411 } }
I20260807 22:43:01.595866 2358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:01.595908 2358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:01.595973 2358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:01.596724 2358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96a0f742604944abe99cc353e8749a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41411 } }
I20260807 22:43:01.597000 2358 leader_election.cc:302] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b96a0f742604944abe99cc353e8749a; no voters:
I20260807 22:43:01.597249 2358 leader_election.cc:288] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:01.597344 2365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:01.597553 2365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [term 1 LEADER]: Becoming Leader. State: Replica: 5b96a0f742604944abe99cc353e8749a, State: Running, Role: LEADER
I20260807 22:43:01.597831 2365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96a0f742604944abe99cc353e8749a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41411 } }
I20260807 22:43:01.598351 2358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:01.599432 2367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b96a0f742604944abe99cc353e8749a. Latest consensus state: current_term: 1 leader_uuid: "5b96a0f742604944abe99cc353e8749a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96a0f742604944abe99cc353e8749a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41411 } } }
I20260807 22:43:01.599562 2367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:01.599850 2366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b96a0f742604944abe99cc353e8749a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96a0f742604944abe99cc353e8749a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41411 } } }
I20260807 22:43:01.599916 2378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:01.600430 2366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:01.603417 2378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:01.608736 2378 catalog_manager.cc:1347] Generated new cluster ID: 3c9ae3d5dc1b424ca0d89976a0d54aa6
I20260807 22:43:01.608803 2378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:01.625742 2378 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:01.626039 2378 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:01.631546 2378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a: Generated new TSK 0
I20260807 22:43:01.632266 2378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:01.642045 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:41411
--builtin_ntp_servers=127.26.189.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:43:01.750756 2394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:01.750990 2394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:01.751029 2394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:01.754926 2394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:01.754971 2394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:01.755028 2394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:01.759649 2394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:01.760607 2394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:01.761904 2394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:01.770852 2411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:01.770984 2409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.771051 2394 server_base.cc:1034] running on GCE node
W20260807 22:43:01.770860 2408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.771515 2394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:01.772156 2394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:01.773334 2394 hybrid_clock.cc:648] HybridClock initialized: now 1786142581773310 us; error 33 us; skew 500 ppm
I20260807 22:43:01.775156 2394 webserver.cc:460] Webserver started at http://127.26.189.193:36849/ using document root <none> and password file <none>
I20260807 22:43:01.775696 2394 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:01.775764 2394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:01.775957 2394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:01.778561 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "fefb454b3dd241c886fe3f026c79e275"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "4073d0919b1d40a6e070e80b276ff0db"
server_key_iv: "2b812916c7402f64f2f6d07b1287e75a"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.779201 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "fefb454b3dd241c886fe3f026c79e275"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "4073d0919b1d40a6e070e80b276ff0db"
server_key_iv: "2b812916c7402f64f2f6d07b1287e75a"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.782562 2394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:01.784816 2423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.785912 2394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:01.786010 2394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "fefb454b3dd241c886fe3f026c79e275"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "4073d0919b1d40a6e070e80b276ff0db"
server_key_iv: "2b812916c7402f64f2f6d07b1287e75a"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.786091 2394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:01.790372 2394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:01.790897 2394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:01.791036 2394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:01.791486 2394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:01.792299 2394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:01.792342 2394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.792377 2394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:01.792397 2394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.800266 2394 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:39775
I20260807 22:43:01.800321 2606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:39775 every 8 connection(s)
I20260807 22:43:01.801067 2394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:01.806620 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.802765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37635 (local address 127.26.189.254:41411)
0807 22:43:01.802924 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.802929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.803956 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 22:43:01.805129 (+ 1173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.805131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.805168 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.805249 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.806222 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.806225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.806227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:01.806237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.806254 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:01.806261 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.806374 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.806471 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.806506 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:43:01.807387 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.803141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:41411 (local address 127.26.189.193:37635)
0807 22:43:01.803793 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.803832 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.804958 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.805383 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.805389 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.805468 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:01.805985 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:01.805987 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:01.806024 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:01.806090 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:01.806364 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:01.806369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:01.806373 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.806501 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.806698 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260807 22:43:01.808475 2609 heartbeater.cc:347] Connected to a master server at 127.26.189.254:41411
I20260807 22:43:01.808719 2609 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:01.808743 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 2394
I20260807 22:43:01.808856 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:01.809156 2609 heartbeater.cc:511] Master 127.26.189.254:41411 requested a full tablet report, sending...
I20260807 22:43:01.809182 27383 external_mini_cluster.cc:1428] Setting key 6a59fabbb1376a8cca5ac2210d45daf1
I20260807 22:43:01.810642 2296 ts_manager.cc:194] Registered new tserver with Master: fefb454b3dd241c886fe3f026c79e275 (127.26.189.193:39775)
I20260807 22:43:01.810997 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:41411
--builtin_ntp_servers=127.26.189.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:43:01.918099 2614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:01.918453 2614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:01.918560 2614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:01.922600 2614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:01.922657 2614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:01.922730 2614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:01.927573 2614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:01.928557 2614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:01.929872 2614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:01.939159 2623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:01.939244 2630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.939262 2614 server_base.cc:1034] running on GCE node
W20260807 22:43:01.939355 2637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:01.939793 2614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:01.940385 2614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:01.941536 2614 hybrid_clock.cc:648] HybridClock initialized: now 1786142581941518 us; error 35 us; skew 500 ppm
I20260807 22:43:01.943225 2614 webserver.cc:460] Webserver started at http://127.26.189.194:44843/ using document root <none> and password file <none>
I20260807 22:43:01.943739 2614 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:01.943804 2614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:01.943991 2614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:01.945912 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "e389175449d947c4a72f2f071cf5a990"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "becd7b4a45aa7718cf1429a2440a05bb"
server_key_iv: "c252ab30afd9fc22cdf119e07ad43c7a"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.946485 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "e389175449d947c4a72f2f071cf5a990"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "becd7b4a45aa7718cf1429a2440a05bb"
server_key_iv: "c252ab30afd9fc22cdf119e07ad43c7a"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.949841 2614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:43:01.952178 2645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.953299 2614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:01.953413 2614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "e389175449d947c4a72f2f071cf5a990"
format_stamp: "Formatted at 2026-08-07 22:43:01 on dist-test-slave-4tdj"
server_key: "becd7b4a45aa7718cf1429a2440a05bb"
server_key_iv: "c252ab30afd9fc22cdf119e07ad43c7a"
server_key_version: "encryptionkey@0"
I20260807 22:43:01.953503 2614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:01.957753 2614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:01.958350 2614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:01.958496 2614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:01.958987 2614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:01.959836 2614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:01.959883 2614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.959931 2614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:01.959957 2614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:01.967998 2614 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:41089
I20260807 22:43:01.968021 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:41089 every 8 connection(s)
I20260807 22:43:01.968838 2614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:01.974364 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.970438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:44033 (local address 127.26.189.254:41411)
0807 22:43:01.970557 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:01.970560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:01.971548 (+ 988us) server_negotiation.cc:407] Connection header received
0807 22:43:01.972788 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:01.972791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:01.972838 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:01.972909 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:01.973921 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:01.973925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:01.973928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:01.973941 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:01.973961 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:01.973968 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:01.974046 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:01.974165 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 22:43:01.974209 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:43:01.975111 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:01.970794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:41411 (local address 127.26.189.194:44033)
0807 22:43:01.971411 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:01.971450 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:01.972574 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:01.973077 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:01.973084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:01.973169 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:01.973672 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:01.973674 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:01.973711 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:01.973776 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:01.974038 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:01.974046 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:01.974052 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:01.974199 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:43:01.974438 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
I20260807 22:43:01.976033 2844 heartbeater.cc:347] Connected to a master server at 127.26.189.254:41411
I20260807 22:43:01.976231 2844 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:01.976585 2844 heartbeater.cc:511] Master 127.26.189.254:41411 requested a full tablet report, sending...
I20260807 22:43:01.976907 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 2614
I20260807 22:43:01.977020 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:01.977316 27383 external_mini_cluster.cc:1428] Setting key 94e751606f805d32e53e03886e202f91
I20260807 22:43:01.977562 2296 ts_manager.cc:194] Registered new tserver with Master: e389175449d947c4a72f2f071cf5a990 (127.26.189.194:41089)
I20260807 22:43:01.979199 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:41411
--builtin_ntp_servers=127.26.189.212:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:43:02.085893 2850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:02.086130 2850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:02.086181 2850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:02.090092 2850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:02.090145 2850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:02.090219 2850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:02.094898 2850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:02.095925 2850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:02.097651 2850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:43:02.109550 2850 server_base.cc:1034] running on GCE node
W20260807 22:43:02.109503 2861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:02.109503 2866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:02.109511 2859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:02.110095 2850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:02.110716 2850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:02.111866 2850 hybrid_clock.cc:648] HybridClock initialized: now 1786142582111853 us; error 23 us; skew 500 ppm
I20260807 22:43:02.113520 2850 webserver.cc:460] Webserver started at http://127.26.189.195:40391/ using document root <none> and password file <none>
I20260807 22:43:02.114050 2850 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:02.114113 2850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:02.114336 2850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:02.116266 2850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "4329bc34d6ea4a5abe7d79773389b158"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "d0b577d2871e4e1cde958d166280930d"
server_key_iv: "1069781e88cbd52fd61fac74bfc5e8d2"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.116799 2850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "4329bc34d6ea4a5abe7d79773389b158"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "d0b577d2871e4e1cde958d166280930d"
server_key_iv: "1069781e88cbd52fd61fac74bfc5e8d2"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.120235 2850 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:02.122515 2875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:02.123622 2850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:02.123718 2850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "4329bc34d6ea4a5abe7d79773389b158"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "d0b577d2871e4e1cde958d166280930d"
server_key_iv: "1069781e88cbd52fd61fac74bfc5e8d2"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.123811 2850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:02.128019 2850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:02.128543 2850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:02.128691 2850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:02.129173 2850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:02.129989 2850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:02.130033 2850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:02.130082 2850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:02.130111 2850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:02.138253 2850 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:33893
I20260807 22:43:02.138330 3049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:33893 every 8 connection(s)
I20260807 22:43:02.139058 2850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:02.144395 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.140666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:55863 (local address 127.26.189.254:41411)
0807 22:43:02.140802 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.140806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.141741 (+ 935us) server_negotiation.cc:407] Connection header received
0807 22:43:02.142976 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.142979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.143017 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.143093 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.144051 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.144053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.144055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.144064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.144079 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.144085 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.144163 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.144256 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.144291 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:43:02.145156 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.140960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:41411 (local address 127.26.189.195:55863)
0807 22:43:02.141597 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.141636 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.142805 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.143221 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.143227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.143307 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:02.143821 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:02.143823 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:02.143860 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:02.143924 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:02.144160 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:02.144165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:02.144168 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.144282 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.144460 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260807 22:43:02.145793 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 2850
I20260807 22:43:02.145902 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:02.146170 27383 external_mini_cluster.cc:1428] Setting key fa9f5df8ad346436f4bfa73c48aab927
I20260807 22:43:02.146313 3050 heartbeater.cc:347] Connected to a master server at 127.26.189.254:41411
I20260807 22:43:02.146528 3050 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:02.146950 3050 heartbeater.cc:511] Master 127.26.189.254:41411 requested a full tablet report, sending...
I20260807 22:43:02.147933 2297 ts_manager.cc:194] Registered new tserver with Master: 4329bc34d6ea4a5abe7d79773389b158 (127.26.189.195:33893)
I20260807 22:43:02.159651 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:43:02.162829 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.161386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.26.189.254:41411)
0807 22:43:02.161511 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.161515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.161659 (+ 144us) server_negotiation.cc:407] Connection header received
0807 22:43:02.161776 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.161779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.161815 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.161876 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.162484 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.162488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.162491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.162502 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.162522 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.162530 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.162607 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.162671 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.162710 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:43:02.165170 2297 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:02.168155 2297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:02.179138 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.176510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33893 (local address 127.0.0.1:53464)
0807 22:43:02.176829 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.176841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.176924 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.177496 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.177499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.177510 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:02.178337 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:02.178339 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:02.178347 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:02.178384 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:02.178838 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:02.178842 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:02.178846 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.178912 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.178979 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":118,"threads_started":1}
I20260807 22:43:02.179167 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.175843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:39775 (local address 127.0.0.1:33418)
0807 22:43:02.176312 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.176359 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.176459 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.177430 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.177437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.177511 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:02.178272 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:02.178274 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:02.178318 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:02.178385 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:02.178907 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:02.178910 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:02.178913 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.178984 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.179009 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":70,"threads_started":1}
I20260807 22:43:02.179220 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.176536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.26.189.194:41089)
0807 22:43:02.177284 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.177296 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.177320 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:43:02.177371 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.177376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.177552 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.177700 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.178648 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.178655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.178664 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.178674 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.178708 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.178727 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.178819 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.179004 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.179075 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":173,"threads_started":1}
I20260807 22:43:02.179533 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.176017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.26.189.193:39775)
0807 22:43:02.176958 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.176970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.176996 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:43:02.177051 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.177056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.177259 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.177781 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.178702 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.178707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.178715 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.178725 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.178765 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.178791 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.178918 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.179312 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.179390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":114,"threads_started":1}
I20260807 22:43:02.179167 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.176199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:41089 (local address 127.0.0.1:37446)
0807 22:43:02.176641 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.176653 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.176727 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.177694 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.177697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.177713 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:02.178500 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:02.178502 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:02.178511 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:02.178535 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:02.178804 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:02.178814 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:02.178820 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.178926 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.178981 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":87,"threads_started":1}
I20260807 22:43:02.179616 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.176800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.26.189.195:33893)
0807 22:43:02.177145 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.177159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.177179 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:43:02.177224 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.177229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.177372 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.177781 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.178636 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.178642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.178653 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.178664 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.178703 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.178729 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.179358 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.179406 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.179468 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":143,"threads_started":1}
I20260807 22:43:02.181982 2498 tablet_service.cc:1467] Processing CreateTablet for tablet 1e28287d344c42c3a1a615cac943a7a8 (DEFAULT_TABLE table=test-table [id=2b8ccd1255b24f2aaced5fe3d4fa0908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:02.181980 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 1e28287d344c42c3a1a615cac943a7a8 (DEFAULT_TABLE table=test-table [id=2b8ccd1255b24f2aaced5fe3d4fa0908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:02.182886 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e28287d344c42c3a1a615cac943a7a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:02.182969 2498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e28287d344c42c3a1a615cac943a7a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:02.184111 2731 tablet_service.cc:1467] Processing CreateTablet for tablet 1e28287d344c42c3a1a615cac943a7a8 (DEFAULT_TABLE table=test-table [id=2b8ccd1255b24f2aaced5fe3d4fa0908]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:02.185010 2731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e28287d344c42c3a1a615cac943a7a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:02.188410 3075 tablet_bootstrap.cc:492] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Bootstrap starting.
I20260807 22:43:02.190284 3076 tablet_bootstrap.cc:492] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: Bootstrap starting.
I20260807 22:43:02.190883 3075 tablet_bootstrap.cc:654] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:02.191782 3075 log.cc:824] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:02.191934 3077 tablet_bootstrap.cc:492] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Bootstrap starting.
I20260807 22:43:02.192564 3076 tablet_bootstrap.cc:654] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:02.193207 3076 log.cc:824] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:02.193884 3075 tablet_bootstrap.cc:492] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: No bootstrap required, opened a new log
I20260807 22:43:02.194005 3077 tablet_bootstrap.cc:654] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:02.194077 3075 ts_tablet_manager.cc:1397] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:43:02.194811 3076 tablet_bootstrap.cc:492] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: No bootstrap required, opened a new log
I20260807 22:43:02.194979 3076 ts_tablet_manager.cc:1397] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260807 22:43:02.194823 3077 log.cc:824] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:02.197080 3077 tablet_bootstrap.cc:492] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: No bootstrap required, opened a new log
I20260807 22:43:02.197259 3077 ts_tablet_manager.cc:1397] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 22:43:02.197775 3075 raft_consensus.cc:348] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.197996 3075 raft_consensus.cc:374] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:02.198050 3075 raft_consensus.cc:729] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4329bc34d6ea4a5abe7d79773389b158, State: Initialized, Role: FOLLOWER
I20260807 22:43:02.198510 3075 consensus_queue.cc:260] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.199141 3050 heartbeater.cc:503] Master 127.26.189.254:41411 was elected leader, sending a full tablet report...
I20260807 22:43:02.199177 3076 raft_consensus.cc:348] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.199412 3076 raft_consensus.cc:374] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:02.199518 3076 raft_consensus.cc:729] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fefb454b3dd241c886fe3f026c79e275, State: Initialized, Role: FOLLOWER
I20260807 22:43:02.199937 3076 consensus_queue.cc:260] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.200536 3075 ts_tablet_manager.cc:1428] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:43:02.200927 3076 ts_tablet_manager.cc:1428] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 22:43:02.201392 2609 heartbeater.cc:503] Master 127.26.189.254:41411 was elected leader, sending a full tablet report...
I20260807 22:43:02.201483 3077 raft_consensus.cc:348] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.201721 3077 raft_consensus.cc:374] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:02.201769 3077 raft_consensus.cc:729] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e389175449d947c4a72f2f071cf5a990, State: Initialized, Role: FOLLOWER
I20260807 22:43:02.202176 3077 consensus_queue.cc:260] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.202765 2844 heartbeater.cc:503] Master 127.26.189.254:41411 was elected leader, sending a full tablet report...
I20260807 22:43:02.203070 3077 ts_tablet_manager.cc:1428] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260807 22:43:02.219739 2845 tablet.cc:2369] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:02.271203 3087 raft_consensus.cc:482] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:02.271404 3087 raft_consensus.cc:504] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.272500 3087 leader_election.cc:288] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4329bc34d6ea4a5abe7d79773389b158 (127.26.189.195:33893), e389175449d947c4a72f2f071cf5a990 (127.26.189.194:41089)
I20260807 22:43:02.274375 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.272761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33893 (local address 127.26.189.193:42217)
0807 22:43:02.272949 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.272965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.273052 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.273280 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.273283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.273298 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:02.273805 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:02.273807 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:02.273820 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:02.273848 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:02.274108 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:02.274112 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:02.274115 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.274190 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.274225 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 22:43:02.274494 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.272914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:42217 (local address 127.26.189.195:33893)
0807 22:43:02.273046 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.273050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.273067 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:02.273146 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.273148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.273180 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.273242 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.273975 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.273979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.273981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.273994 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.274018 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.274029 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.274121 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.274324 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.274371 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:43:02.274880 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.272785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:41089 (local address 127.26.189.193:33985)
0807 22:43:02.273218 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.273229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.273319 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.273618 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.273621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.273635 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:02.274205 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:02.274207 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:02.274218 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:02.274241 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:02.274640 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:02.274643 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:02.274647 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:43:02.274718 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:43:02.274747 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":164,"threads_started":1}
I20260807 22:43:02.274899 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.272894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:33985 (local address 127.26.189.194:41089)
0807 22:43:02.273097 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.273102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.273309 (+ 207us) server_negotiation.cc:407] Connection header received
0807 22:43:02.273401 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.273404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.273437 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.273511 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.274378 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.274381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.274384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.274396 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.274418 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.274428 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.274513 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.274739 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.274783 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 22:43:02.275115 2981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e28287d344c42c3a1a615cac943a7a8" candidate_uuid: "fefb454b3dd241c886fe3f026c79e275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4329bc34d6ea4a5abe7d79773389b158" is_pre_election: true
I20260807 22:43:02.275432 2981 raft_consensus.cc:2454] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fefb454b3dd241c886fe3f026c79e275 in term 0.
I20260807 22:43:02.275485 2761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e28287d344c42c3a1a615cac943a7a8" candidate_uuid: "fefb454b3dd241c886fe3f026c79e275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e389175449d947c4a72f2f071cf5a990" is_pre_election: true
I20260807 22:43:02.275767 2761 raft_consensus.cc:2454] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fefb454b3dd241c886fe3f026c79e275 in term 0.
I20260807 22:43:02.275959 2429 leader_election.cc:302] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4329bc34d6ea4a5abe7d79773389b158, fefb454b3dd241c886fe3f026c79e275; no voters:
I20260807 22:43:02.276249 3087 raft_consensus.cc:2781] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:02.276340 3087 raft_consensus.cc:482] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:02.276389 3087 raft_consensus.cc:3037] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:02.277431 3087 raft_consensus.cc:504] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.277742 3087 leader_election.cc:288] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [CANDIDATE]: Term 1 election: Requested vote from peers 4329bc34d6ea4a5abe7d79773389b158 (127.26.189.195:33893), e389175449d947c4a72f2f071cf5a990 (127.26.189.194:41089)
I20260807 22:43:02.278069 2981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e28287d344c42c3a1a615cac943a7a8" candidate_uuid: "fefb454b3dd241c886fe3f026c79e275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4329bc34d6ea4a5abe7d79773389b158"
I20260807 22:43:02.278098 2761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e28287d344c42c3a1a615cac943a7a8" candidate_uuid: "fefb454b3dd241c886fe3f026c79e275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e389175449d947c4a72f2f071cf5a990"
I20260807 22:43:02.278208 2761 raft_consensus.cc:3037] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:02.278203 2981 raft_consensus.cc:3037] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:02.279526 2761 raft_consensus.cc:2454] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fefb454b3dd241c886fe3f026c79e275 in term 1.
I20260807 22:43:02.279526 2981 raft_consensus.cc:2454] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fefb454b3dd241c886fe3f026c79e275 in term 1.
I20260807 22:43:02.279822 2425 leader_election.cc:302] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e389175449d947c4a72f2f071cf5a990, fefb454b3dd241c886fe3f026c79e275; no voters:
I20260807 22:43:02.279978 3087 raft_consensus.cc:2781] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:02.280201 3087 raft_consensus.cc:686] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 1 LEADER]: Becoming Leader. State: Replica: fefb454b3dd241c886fe3f026c79e275, State: Running, Role: LEADER
I20260807 22:43:02.280525 3087 consensus_queue.cc:237] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } }
I20260807 22:43:02.283558 2297 catalog_manager.cc:5495] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 reported cstate change: term changed from 0 to 1, leader changed from <none> to fefb454b3dd241c886fe3f026c79e275 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "fefb454b3dd241c886fe3f026c79e275" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fefb454b3dd241c886fe3f026c79e275" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 39775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 } health_report { overall_health: UNKNOWN } } }
W20260807 22:43:02.302196 2612 tablet.cc:2369] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:02.304035 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.302790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33428 (local address 127.26.189.193:39775)
0807 22:43:02.302902 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.302907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.302921 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:43:02.303040 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.303042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.303104 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.303175 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.303741 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.303744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.303747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.303758 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.303777 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.303785 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.303858 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.303902 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.303935 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:43:02.309811 2981 raft_consensus.cc:1261] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 1 FOLLOWER]: Refusing update from remote peer fefb454b3dd241c886fe3f026c79e275: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:02.309811 2761 raft_consensus.cc:1261] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 1 FOLLOWER]: Refusing update from remote peer fefb454b3dd241c886fe3f026c79e275: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:02.310532 3092 consensus_queue.cc:1035] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e389175449d947c4a72f2f071cf5a990" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 41089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:02.310706 3087 consensus_queue.cc:1035] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4329bc34d6ea4a5abe7d79773389b158" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:02.317616 3101 mvcc.cc:205] Tried to move back new op lower bound from 7316040017133682688 to 7316040017023614976. Current Snapshot: MvccSnapshot[applied={T|T < 7316040017133682688}]
I20260807 22:43:02.318037 3100 mvcc.cc:205] Tried to move back new op lower bound from 7316040017133682688 to 7316040017023614976. Current Snapshot: MvccSnapshot[applied={T|T < 7316040017133682688}]
I20260807 22:43:02.319098 3098 mvcc.cc:205] Tried to move back new op lower bound from 7316040017133682688 to 7316040017023614976. Current Snapshot: MvccSnapshot[applied={T|T < 7316040017133682688}]
I20260807 22:43:02.326603 2296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:02.326757 2296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:02.329174 2296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b96a0f742604944abe99cc353e8749a: Sending DeleteTablet for 3 replicas of tablet 1e28287d344c42c3a1a615cac943a7a8
I20260807 22:43:02.329968 2498 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e28287d344c42c3a1a615cac943a7a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:02 UTC) from {username='slave'} at 127.0.0.1:33418
I20260807 22:43:02.330197 2731 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e28287d344c42c3a1a615cac943a7a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:02 UTC) from {username='slave'} at 127.0.0.1:37446
I20260807 22:43:02.330523 3115 tablet_replica.cc:330] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275: stopping tablet replica
I20260807 22:43:02.330646 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e28287d344c42c3a1a615cac943a7a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:02 UTC) from {username='slave'} at 127.0.0.1:53464
I20260807 22:43:02.330950 3116 tablet_replica.cc:330] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: stopping tablet replica
I20260807 22:43:02.330964 3115 raft_consensus.cc:2229] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:02.331190 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 2394
I20260807 22:43:02.331254 3116 raft_consensus.cc:2229] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:02.331542 3116 raft_consensus.cc:2258] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:02.331532 3115 raft_consensus.cc:2258] T 1e28287d344c42c3a1a615cac943a7a8 P fefb454b3dd241c886fe3f026c79e275 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:02.335239 3116 ts_tablet_manager.cc:1905] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:02.337743 3117 tablet_replica.cc:330] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: stopping tablet replica
I20260807 22:43:02.338085 3117 raft_consensus.cc:2229] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:02.338312 3116 ts_tablet_manager.cc:1918] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:02.338408 3116 log.cc:1196] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/1e28287d344c42c3a1a615cac943a7a8
I20260807 22:43:02.338454 3117 raft_consensus.cc:2258] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:02.338758 3116 ts_tablet_manager.cc:1939] T 1e28287d344c42c3a1a615cac943a7a8 P e389175449d947c4a72f2f071cf5a990: Deleting consensus metadata
I20260807 22:43:02.339866 2270 catalog_manager.cc:4841] TS e389175449d947c4a72f2f071cf5a990 (127.26.189.194:41089): tablet 1e28287d344c42c3a1a615cac943a7a8 (table test-table [id=2b8ccd1255b24f2aaced5fe3d4fa0908]) successfully deleted
I20260807 22:43:02.340826 3117 ts_tablet_manager.cc:1905] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:02.342912 3117 ts_tablet_manager.cc:1918] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:02.342993 3117 log.cc:1196] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/1e28287d344c42c3a1a615cac943a7a8
I20260807 22:43:02.343240 3117 ts_tablet_manager.cc:1939] T 1e28287d344c42c3a1a615cac943a7a8 P 4329bc34d6ea4a5abe7d79773389b158: Deleting consensus metadata
I20260807 22:43:02.344247 2275 catalog_manager.cc:4841] TS 4329bc34d6ea4a5abe7d79773389b158 (127.26.189.195:33893): tablet 1e28287d344c42c3a1a615cac943a7a8 (table test-table [id=2b8ccd1255b24f2aaced5fe3d4fa0908]) successfully deleted
W20260807 22:43:02.345434 2275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:39775 (error 108)
I20260807 22:43:02.345582 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 2614
I20260807 22:43:02.347705 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.346216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:39775 (local address 127.0.0.1:33430)
0807 22:43:02.347542 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.347623 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:39775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1229}
W20260807 22:43:02.347860 2275 catalog_manager.cc:4584] TS fefb454b3dd241c886fe3f026c79e275 (127.26.189.193:39775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e28287d344c42c3a1a615cac943a7a8: Network error: Client connection negotiation failed: client connection to 127.26.189.193:39775: connect: Connection refused (error 111)
I20260807 22:43:02.356853 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 2850
I20260807 22:43:02.366880 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 2245
2026-08-07T22:43:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (973 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 22:43:02.389225 27383 mini_oidc.cc:152] Starting JWKS server
I20260807 22:43:02.391021 27383 webserver.cc:460] Webserver started at https://127.0.0.1:42365/ using document root <none> and password file <none>
I20260807 22:43:02.391192 27383 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:43:02.391433 27383 webserver.cc:460] Webserver started at http://127.0.0.1:46373/ using document root <none> and password file <none>
2026-08-07T22:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:02Z Disabled control of system clock
I20260807 22:43:02.400570 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:45251
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:39869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:45251
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42365/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/testchainca.cert with env {}
W20260807 22:43:02.507454 3134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:02.507686 3134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:02.507725 3134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:02.511588 3134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:02.511633 3134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:02.511647 3134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:02.511662 3134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:02.511680 3134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:02.511695 3134 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:02.511709 3134 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42365/jwks.json
I20260807 22:43:02.516855 3134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:45251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42365/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:02.517778 3134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:02.518707 3134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:02.526814 3149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:02.526825 3145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:02.526988 3134 server_base.cc:1034] running on GCE node
W20260807 22:43:02.526829 3146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:02.527840 3134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:02.528843 3134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:02.530000 3134 hybrid_clock.cc:648] HybridClock initialized: now 1786142582529976 us; error 38 us; skew 500 ppm
I20260807 22:43:02.531698 3134 webserver.cc:460] Webserver started at http://127.26.189.254:42031/ using document root <none> and password file <none>
I20260807 22:43:02.532244 3134 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:02.532311 3134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:02.532502 3134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:02.534475 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "5f648e6a2c65431aa9bfba969085887c"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "718c16473307e69e214e818f81979bb6"
server_key_iv: "f05c7e8566a0c35447e3b7cdafca8494"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.534992 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "5f648e6a2c65431aa9bfba969085887c"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "718c16473307e69e214e818f81979bb6"
server_key_iv: "f05c7e8566a0c35447e3b7cdafca8494"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.538444 3134 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:43:02.540657 3163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:02.541769 3134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:02.541887 3134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "5f648e6a2c65431aa9bfba969085887c"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "718c16473307e69e214e818f81979bb6"
server_key_iv: "f05c7e8566a0c35447e3b7cdafca8494"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.541975 3134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:02.552130 3134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:02.552695 3134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:02.552846 3134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:02.559226 3134 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:45251
I20260807 22:43:02.559232 3241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:45251 every 8 connection(s)
I20260807 22:43:02.560053 3134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:02.562943 3243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:02.565830 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 3134
I20260807 22:43:02.565941 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:02.566195 27383 external_mini_cluster.cc:1428] Setting key 5ba63c6d192dccb40b64aba5abbdb19c
I20260807 22:43:02.568534 3243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c: Bootstrap starting.
I20260807 22:43:02.570677 3243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:02.571378 3243 log.cc:824] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:02.573165 3243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c: No bootstrap required, opened a new log
I20260807 22:43:02.574860 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.567484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.26.189.254:45251)
0807 22:43:02.568102 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.568116 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.568151 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:43:02.568825 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.568843 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.569220 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.569507 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.569938 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.570942 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.571825 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.572269 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.572412 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.573206 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.573211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.573223 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.573607 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.573660 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.573681 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.573768 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.573905 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.574148 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":138,"threads_started":1}
I20260807 22:43:02.576594 3243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f648e6a2c65431aa9bfba969085887c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45251 } }
I20260807 22:43:02.576835 3243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:02.576898 3243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f648e6a2c65431aa9bfba969085887c, State: Initialized, Role: FOLLOWER
I20260807 22:43:02.577276 3243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f648e6a2c65431aa9bfba969085887c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45251 } }
I20260807 22:43:02.577378 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:02.577438 3243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:02.577504 3243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:02.577598 3243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:02.578480 3243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f648e6a2c65431aa9bfba969085887c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45251 } }
I20260807 22:43:02.578783 3243 leader_election.cc:302] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f648e6a2c65431aa9bfba969085887c; no voters:
I20260807 22:43:02.579056 3243 leader_election.cc:288] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:02.579145 3248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:02.579349 3248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [term 1 LEADER]: Becoming Leader. State: Replica: 5f648e6a2c65431aa9bfba969085887c, State: Running, Role: LEADER
I20260807 22:43:02.579648 3248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f648e6a2c65431aa9bfba969085887c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45251 } }
I20260807 22:43:02.580278 3243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:02.581286 3254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f648e6a2c65431aa9bfba969085887c. Latest consensus state: current_term: 1 leader_uuid: "5f648e6a2c65431aa9bfba969085887c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f648e6a2c65431aa9bfba969085887c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45251 } } }
I20260807 22:43:02.581278 3250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f648e6a2c65431aa9bfba969085887c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f648e6a2c65431aa9bfba969085887c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45251 } } }
I20260807 22:43:02.581506 3254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:02.581506 3250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:02.581880 3265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:02.584547 3265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:02.589550 3265 catalog_manager.cc:1347] Generated new cluster ID: 18715374532b407f8bf0fa2cdc5eabc2
I20260807 22:43:02.589613 3265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:02.613059 3265 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:02.613894 3265 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:02.622506 3265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f648e6a2c65431aa9bfba969085887c: Generated new TSK 0
I20260807 22:43:02.623117 3265 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:43:02.653455 3298 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 22:43:02.653617 3298 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:43:02.654068 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.636912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.26.189.254:45251)
0807 22:43:02.637038 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.637044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.637262 (+ 218us) server_negotiation.cc:407] Connection header received
0807 22:43:02.637457 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.637459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.637505 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.637579 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:02.637834 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.638790 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.640099 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.640315 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.640385 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.640708 (+ 323us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:02.653741 (+ 13033us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:43:02.653962 (+ 221us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51188: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 22:43:02.654139 3246 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51188: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 22:43:02.655575 3294 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:02.636857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45251 (local address 127.0.0.1:51188)
0807 22:43:02.637182 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.637197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.637322 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.637609 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.637612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.637620 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:02.637745 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:02.637751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.638951 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.638956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:02.639993 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:02.640004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.640406 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.640410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:02.640503 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.640554 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:02.653951 (+ 13397us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:43:02.654191 (+ 240us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.189.254:45251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":103,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.189.254:45251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 22:43:02.656237 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 3134
2026-08-07T22:43:02Z chronyd exiting
I20260807 22:43:02.704370 27383 test_util.cc:175] -----------------------------------------------
I20260807 22:43:02.704452 27383 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142556986508-27383-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (316 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 22:43:02.705596 27383 mini_oidc.cc:152] Starting JWKS server
I20260807 22:43:02.706452 27383 webserver.cc:460] Webserver started at https://127.0.0.1:33689/ using document root <none> and password file <none>
I20260807 22:43:02.706548 27383 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:43:02.706806 27383 webserver.cc:460] Webserver started at http://127.0.0.1:37331/ using document root <none> and password file <none>
2026-08-07T22:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:02Z Disabled control of system clock
I20260807 22:43:02.715950 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36309
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36309
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33689/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/testchainca.cert with env {}
W20260807 22:43:02.821786 3309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:02.822021 3309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:02.822057 3309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:02.825937 3309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:02.825987 3309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:02.826001 3309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:02.826030 3309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:02.826056 3309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:02.826085 3309 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:02.826112 3309 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33689/jwks.json
I20260807 22:43:02.831243 3309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33689/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:02.832181 3309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:02.833142 3309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:02.841046 3332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:02.841053 3322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:02.841053 3324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:02.841382 3309 server_base.cc:1034] running on GCE node
I20260807 22:43:02.841889 3309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:02.842964 3309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:02.844122 3309 hybrid_clock.cc:648] HybridClock initialized: now 1786142582844112 us; error 32 us; skew 500 ppm
I20260807 22:43:02.845788 3309 webserver.cc:460] Webserver started at http://127.26.189.254:37349/ using document root <none> and password file <none>
I20260807 22:43:02.846350 3309 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:02.846423 3309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:02.846621 3309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:02.848515 3309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "e20c7b0096a74e6cb36a1dd57ef4756b"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "e19aad7a9f6a393092cbb11924de2649"
server_key_iv: "93612079573b4e45d02992523ed2fe7d"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.849062 3309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "e20c7b0096a74e6cb36a1dd57ef4756b"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "e19aad7a9f6a393092cbb11924de2649"
server_key_iv: "93612079573b4e45d02992523ed2fe7d"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.852417 3309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 22:43:02.854652 3340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:02.855738 3309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:02.855845 3309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "e20c7b0096a74e6cb36a1dd57ef4756b"
format_stamp: "Formatted at 2026-08-07 22:43:02 on dist-test-slave-4tdj"
server_key: "e19aad7a9f6a393092cbb11924de2649"
server_key_iv: "93612079573b4e45d02992523ed2fe7d"
server_key_version: "encryptionkey@0"
I20260807 22:43:02.855934 3309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:02.869318 3309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:02.869952 3309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:02.870119 3309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:02.876582 3309 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:36309
I20260807 22:43:02.876593 3428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:36309 every 8 connection(s)
I20260807 22:43:02.877475 3309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:02.880553 3429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:02.880877 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 3309
I20260807 22:43:02.880997 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:02.881279 27383 external_mini_cluster.cc:1428] Setting key cbb08750b540131ab8e19b330ef40c63
I20260807 22:43:02.887356 3429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b: Bootstrap starting.
I20260807 22:43:02.889811 3429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:02.890715 3429 log.cc:824] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:02.891844 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.883126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.26.189.254:36309)
0807 22:43:02.883802 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.883821 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.883867 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:43:02.884686 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.884710 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.885192 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.885574 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:02.886023 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.887467 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.888328 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.889000 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.889151 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.889852 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:02.889859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:02.889875 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:02.890378 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:02.890446 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:02.890472 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:02.890571 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:02.890691 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 22:43:02.890947 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":96,"threads_started":1}
I20260807 22:43:02.892736 3429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b: No bootstrap required, opened a new log
I20260807 22:43:02.894573 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:02.896612 3429 raft_consensus.cc:348] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36309 } }
I20260807 22:43:02.896826 3429 raft_consensus.cc:374] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:02.896881 3429 raft_consensus.cc:729] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20c7b0096a74e6cb36a1dd57ef4756b, State: Initialized, Role: FOLLOWER
I20260807 22:43:02.897296 3429 consensus_queue.cc:260] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36309 } }
I20260807 22:43:02.897430 3429 raft_consensus.cc:388] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:02.897495 3429 raft_consensus.cc:482] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:02.897588 3429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:02.898471 3429 raft_consensus.cc:504] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36309 } }
I20260807 22:43:02.898774 3429 leader_election.cc:302] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e20c7b0096a74e6cb36a1dd57ef4756b; no voters:
I20260807 22:43:02.899048 3429 leader_election.cc:288] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:02.899161 3437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:02.899359 3437 raft_consensus.cc:686] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [term 1 LEADER]: Becoming Leader. State: Replica: e20c7b0096a74e6cb36a1dd57ef4756b, State: Running, Role: LEADER
I20260807 22:43:02.899881 3437 consensus_queue.cc:237] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36309 } }
I20260807 22:43:02.900288 3429 sys_catalog.cc:564] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:02.901342 3440 sys_catalog.cc:455] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e20c7b0096a74e6cb36a1dd57ef4756b. Latest consensus state: current_term: 1 leader_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36309 } } }
I20260807 22:43:02.901471 3440 sys_catalog.cc:458] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:02.901854 3451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:02.901799 3438 sys_catalog.cc:455] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20c7b0096a74e6cb36a1dd57ef4756b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36309 } } }
I20260807 22:43:02.902390 3438 sys_catalog.cc:458] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:02.905195 3451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:02.910156 3451 catalog_manager.cc:1347] Generated new cluster ID: dd74d1e350de40a381436ff1ee9c5662
I20260807 22:43:02.910218 3451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:02.926164 3451 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:02.927027 3451 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:02.931304 3451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e20c7b0096a74e6cb36a1dd57ef4756b: Generated new TSK 0
I20260807 22:43:02.931897 3451 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:43:02.963598 3471 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 22:43:02.963701 3471 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:43:02.964336 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:02.941550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42260 (local address 127.26.189.254:36309)
0807 22:43:02.941694 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:02.941700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:02.941867 (+ 167us) server_negotiation.cc:407] Connection header received
0807 22:43:02.942020 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:02.942022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:02.942068 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:02.942212 (+ 144us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:02.942504 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.943171 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.944371 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.944590 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.944669 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.944969 (+ 300us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:02.963974 (+ 19005us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:43:02.964211 (+ 237us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 22:43:02.964397 3435 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42260: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 22:43:02.964586 3470 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:02.941490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36309 (local address 127.0.0.1:42260)
0807 22:43:02.941779 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:02.941794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:02.941910 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:02.942228 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:02.942231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:02.942238 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:02.942393 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:02.942400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.943277 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.943280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:02.944254 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:02.944264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:02.944692 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:02.944696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:02.944791 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:02.944820 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:02.964187 (+ 19367us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:43:02.964398 (+ 211us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.189.254:36309: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33689/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":112,"threads_started":1}
I20260807 22:43:02.965197 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 3309
2026-08-07T22:43:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (305 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 22:43:03.011345 27383 mini_oidc.cc:152] Starting JWKS server
I20260807 22:43:03.012197 27383 webserver.cc:460] Webserver started at https://127.0.0.1:42599/ using document root <none> and password file <none>
I20260807 22:43:03.012297 27383 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:43:03.012545 27383 webserver.cc:460] Webserver started at http://127.0.0.1:45945/ using document root <none> and password file <none>
2026-08-07T22:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:03Z Disabled control of system clock
I20260807 22:43:03.025161 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43015
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:41943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43015
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42599/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:43:03.132512 3481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:03.132771 3481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:03.132807 3481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:03.136662 3481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:03.136719 3481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:03.136732 3481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:03.136751 3481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:03.136770 3481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:03.136786 3481 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:03.136798 3481 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42599/jwks.json
I20260807 22:43:03.141434 3481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42599/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:03.142460 3481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:03.143764 3481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:03.151746 3490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:03.151796 3492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:03.151777 3488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:03.151911 3481 server_base.cc:1034] running on GCE node
I20260807 22:43:03.152486 3481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:03.153501 3481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:03.154662 3481 hybrid_clock.cc:648] HybridClock initialized: now 1786142583154640 us; error 32 us; skew 500 ppm
I20260807 22:43:03.156378 3481 webserver.cc:460] Webserver started at http://127.26.189.254:39711/ using document root <none> and password file <none>
I20260807 22:43:03.156935 3481 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:03.156998 3481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:03.157199 3481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:03.159231 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "d16f760906e64b0e8c89a49f3b2659b5"
format_stamp: "Formatted at 2026-08-07 22:43:03 on dist-test-slave-4tdj"
server_key: "30c847123b5209f8fe5527b50e78b333"
server_key_iv: "398959716a2bd1e16f8d4e5ed41b0596"
server_key_version: "encryptionkey@0"
I20260807 22:43:03.159759 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "d16f760906e64b0e8c89a49f3b2659b5"
format_stamp: "Formatted at 2026-08-07 22:43:03 on dist-test-slave-4tdj"
server_key: "30c847123b5209f8fe5527b50e78b333"
server_key_iv: "398959716a2bd1e16f8d4e5ed41b0596"
server_key_version: "encryptionkey@0"
I20260807 22:43:03.163218 3481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:03.165531 3499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:03.166788 3481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:03.166890 3481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "d16f760906e64b0e8c89a49f3b2659b5"
format_stamp: "Formatted at 2026-08-07 22:43:03 on dist-test-slave-4tdj"
server_key: "30c847123b5209f8fe5527b50e78b333"
server_key_iv: "398959716a2bd1e16f8d4e5ed41b0596"
server_key_version: "encryptionkey@0"
I20260807 22:43:03.166991 3481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:03.175798 3481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:03.176374 3481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:03.176534 3481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:03.183092 3481 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43015
I20260807 22:43:03.183104 3591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43015 every 8 connection(s)
I20260807 22:43:03.183951 3481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:03.186609 3593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:03.190407 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 3481
I20260807 22:43:03.190521 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:03.190793 27383 external_mini_cluster.cc:1428] Setting key 1ae26d38117823d2d47f0d9f24529919
I20260807 22:43:03.192129 3593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5: Bootstrap starting.
I20260807 22:43:03.194769 3593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:03.195647 3593 log.cc:824] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:03.197494 3593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5: No bootstrap required, opened a new log
I20260807 22:43:03.200567 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:03.192083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.26.189.254:43015)
0807 22:43:03.192722 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:03.192744 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:03.192784 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:43:03.193577 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:03.193600 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:03.194065 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:03.194434 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:03.194868 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.196233 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.197102 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.197747 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.197905 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.198657 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:03.198663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:03.198678 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:03.199155 (+ 477us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:03.199224 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:03.199250 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:03.199349 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:03.199419 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 22:43:03.199674 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260807 22:43:03.201622 3593 raft_consensus.cc:348] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16f760906e64b0e8c89a49f3b2659b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43015 } }
I20260807 22:43:03.201819 3593 raft_consensus.cc:374] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:03.201854 3593 raft_consensus.cc:729] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16f760906e64b0e8c89a49f3b2659b5, State: Initialized, Role: FOLLOWER
I20260807 22:43:03.202310 3593 consensus_queue.cc:260] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16f760906e64b0e8c89a49f3b2659b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43015 } }
I20260807 22:43:03.202474 3593 raft_consensus.cc:388] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:03.202544 3593 raft_consensus.cc:482] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:03.202641 3593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:03.203151 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:03.203533 3593 raft_consensus.cc:504] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16f760906e64b0e8c89a49f3b2659b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43015 } }
I20260807 22:43:03.203889 3593 leader_election.cc:302] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d16f760906e64b0e8c89a49f3b2659b5; no voters:
I20260807 22:43:03.204195 3593 leader_election.cc:288] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:03.204289 3598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:03.204468 3598 raft_consensus.cc:686] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [term 1 LEADER]: Becoming Leader. State: Replica: d16f760906e64b0e8c89a49f3b2659b5, State: Running, Role: LEADER
I20260807 22:43:03.204746 3598 consensus_queue.cc:237] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16f760906e64b0e8c89a49f3b2659b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43015 } }
I20260807 22:43:03.205412 3593 sys_catalog.cc:564] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:03.206269 3601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d16f760906e64b0e8c89a49f3b2659b5. Latest consensus state: current_term: 1 leader_uuid: "d16f760906e64b0e8c89a49f3b2659b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16f760906e64b0e8c89a49f3b2659b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43015 } } }
I20260807 22:43:03.206424 3600 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d16f760906e64b0e8c89a49f3b2659b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d16f760906e64b0e8c89a49f3b2659b5" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43015 } } }
I20260807 22:43:03.206499 3601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:03.206528 3600 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:03.207067 3610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:03.210057 3610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:03.215400 3610 catalog_manager.cc:1347] Generated new cluster ID: c876ad993574496b8cd07b66428219cd
I20260807 22:43:03.215462 3610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:03.229923 3610 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:03.231103 3610 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:03.245230 3610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d16f760906e64b0e8c89a49f3b2659b5: Generated new TSK 0
I20260807 22:43:03.246001 3610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:03.271170 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:03.263819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42294 (local address 127.26.189.254:43015)
0807 22:43:03.263945 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:03.263951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:03.264089 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:43:03.264240 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:03.264243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:03.264304 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:03.264427 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:03.264619 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.265455 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.266313 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.266680 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.266780 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.267039 (+ 259us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:03.270809 (+ 3770us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:43:03.271033 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42294: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 22:43:03.271240 3596 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42294: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 22:43:03.272856 3646 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:03.263723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43015 (local address 127.0.0.1:42294)
0807 22:43:03.264013 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:03.264026 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:03.264120 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:03.264421 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:03.264423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:03.264429 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:03.264527 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:03.264531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.265546 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.265548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:03.266189 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:03.266197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.266791 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.266794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:03.266858 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.266881 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:03.270974 (+ 4093us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:43:03.272686 (+ 1712us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.189.254:43015: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":1391,"thread_start_us":104,"threads_started":1}
I20260807 22:43:03.273509 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 3481
E20260807 22:43:03.282213 3647 webserver.cc:533] Webserver: sslize failed:
E20260807 22:43:03.282277 3647 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T22:43:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (304 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T22:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:03Z Disabled control of system clock
I20260807 22:43:03.325201 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:35457
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:33739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:35457
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/testchainca.cert with env {}
W20260807 22:43:03.431218 3652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:03.431464 3652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:03.431519 3652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:03.435484 3652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:03.435549 3652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:03.435570 3652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:03.435590 3652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:03.435612 3652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:43:03.435640 3652 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:43:03.435660 3652 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 22:43:03.440977 3652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:35457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:35457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:03.442023 3652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:03.443073 3652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:03.451026 3660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:03.451037 3665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:03.451037 3661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:03.451239 3652 server_base.cc:1034] running on GCE node
I20260807 22:43:03.451772 3652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:03.452848 3652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:03.454011 3652 hybrid_clock.cc:648] HybridClock initialized: now 1786142583453992 us; error 29 us; skew 500 ppm
I20260807 22:43:03.455767 3652 webserver.cc:460] Webserver started at http://127.26.189.254:35367/ using document root <none> and password file <none>
I20260807 22:43:03.456305 3652 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:03.456370 3652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:03.456566 3652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:03.458595 3652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "71e9c009f74f456293009f67a845d0f7"
format_stamp: "Formatted at 2026-08-07 22:43:03 on dist-test-slave-4tdj"
server_key: "96a8d8bc245c0d42cdf23099ff5a0875"
server_key_iv: "bc4058ecd86333cad777b5ef3ad17235"
server_key_version: "encryptionkey@0"
I20260807 22:43:03.459178 3652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "71e9c009f74f456293009f67a845d0f7"
format_stamp: "Formatted at 2026-08-07 22:43:03 on dist-test-slave-4tdj"
server_key: "96a8d8bc245c0d42cdf23099ff5a0875"
server_key_iv: "bc4058ecd86333cad777b5ef3ad17235"
server_key_version: "encryptionkey@0"
I20260807 22:43:03.462688 3652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:43:03.464951 3675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:03.466092 3652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:03.466212 3652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "71e9c009f74f456293009f67a845d0f7"
format_stamp: "Formatted at 2026-08-07 22:43:03 on dist-test-slave-4tdj"
server_key: "96a8d8bc245c0d42cdf23099ff5a0875"
server_key_iv: "bc4058ecd86333cad777b5ef3ad17235"
server_key_version: "encryptionkey@0"
I20260807 22:43:03.466332 3652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:03.487510 3652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:03.488200 3652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:03.488371 3652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:03.495123 3652 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:35457
I20260807 22:43:03.495127 3751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:35457 every 8 connection(s)
I20260807 22:43:03.496016 3652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:03.499032 3753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:03.500146 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 3652
I20260807 22:43:03.500260 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:03.500526 27383 external_mini_cluster.cc:1428] Setting key bc82f2960e762768e7d81ab3d570225f
I20260807 22:43:03.504792 3753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7: Bootstrap starting.
I20260807 22:43:03.507004 3753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:03.507762 3753 log.cc:824] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:03.509166 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:03.501813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.26.189.254:35457)
0807 22:43:03.502476 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:03.502492 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:03.502527 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:43:03.503144 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:03.503160 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:03.503525 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:03.503828 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:03.504228 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.505199 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.506052 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.506557 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.506710 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.507533 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:03.507538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:03.507550 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:03.507948 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:03.508003 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:03.508023 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:03.508105 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:03.508249 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 22:43:03.508488 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":134,"threads_started":1}
I20260807 22:43:03.509634 3753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7: No bootstrap required, opened a new log
I20260807 22:43:03.511680 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:43:03.513296 3753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e9c009f74f456293009f67a845d0f7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35457 } }
I20260807 22:43:03.513496 3753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:03.513533 3753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e9c009f74f456293009f67a845d0f7, State: Initialized, Role: FOLLOWER
I20260807 22:43:03.513895 3753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e9c009f74f456293009f67a845d0f7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35457 } }
I20260807 22:43:03.514011 3753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:03.514055 3753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:03.514129 3753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:03.514930 3753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e9c009f74f456293009f67a845d0f7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35457 } }
I20260807 22:43:03.515218 3753 leader_election.cc:302] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71e9c009f74f456293009f67a845d0f7; no voters:
I20260807 22:43:03.515450 3753 leader_election.cc:288] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:03.515563 3761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:03.515815 3761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [term 1 LEADER]: Becoming Leader. State: Replica: 71e9c009f74f456293009f67a845d0f7, State: Running, Role: LEADER
I20260807 22:43:03.516116 3761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e9c009f74f456293009f67a845d0f7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35457 } }
I20260807 22:43:03.516566 3753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:03.517709 3763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71e9c009f74f456293009f67a845d0f7. Latest consensus state: current_term: 1 leader_uuid: "71e9c009f74f456293009f67a845d0f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e9c009f74f456293009f67a845d0f7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35457 } } }
I20260807 22:43:03.517706 3762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71e9c009f74f456293009f67a845d0f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71e9c009f74f456293009f67a845d0f7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35457 } } }
I20260807 22:43:03.517866 3763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:03.517866 3762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:03.518273 3776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:03.520893 3776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:03.525657 3776 catalog_manager.cc:1347] Generated new cluster ID: 4a5dda058fa840998b05c2c06fcb1a45
I20260807 22:43:03.525718 3776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:03.536917 3776 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:03.538070 3776 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:03.546118 3776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71e9c009f74f456293009f67a845d0f7: Generated new TSK 0
I20260807 22:43:03.546902 3776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:04.054512 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:03.561275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.26.189.254:35457)
0807 22:43:03.561402 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:03.561407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:03.561583 (+ 176us) server_negotiation.cc:407] Connection header received
0807 22:43:03.561740 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:03.561742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:03.561788 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:03.561874 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:43:03.562102 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.562722 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.563926 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.564138 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.564201 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.564504 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:43:04.054077 (+489573us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:43:04.054385 (+ 308us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36136: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 22:43:04.054605 3758 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36136: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 22:43:04.054994 3796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:03.561164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:35457 (local address 127.0.0.1:36136)
0807 22:43:03.561496 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:03.561511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:03.561632 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:03.561890 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:03.561893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:03.561901 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:43:03.562012 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:03.562019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.562831 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.562834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:03.563824 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:03.563835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:03.564236 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:03.564240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:03.564333 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:03.564361 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:43:04.054341 (+489980us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:43:04.054525 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.189.254:35457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":103,"threads_started":1}
I20260807 22:43:04.055523 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 3652
2026-08-07T22:43:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (768 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (121 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:04 dist-test-slave-4tdj krb5kdc[3816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:04 dist-test-slave-4tdj krb5kdc[3816](info): set up 2 sockets
Aug 07 22:43:04 dist-test-slave-4tdj krb5kdc[3816](info): commencing operation
krb5kdc: starting...
W20260807 22:43:05.430718 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:05 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:43:05.549021 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36125
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:05.655586 3832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:05.655829 3832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:05.655870 3832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:05.659759 3832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:05.659808 3832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:05.659826 3832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:05.659854 3832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:05.659881 3832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:05.664934 3832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:05.665931 3832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:05.666914 3832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:05.674783 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:05.674823 3846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:05.674805 3845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:05.675019 3832 server_base.cc:1034] running on GCE node
I20260807 22:43:05.675546 3832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:05.676581 3832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:05.677748 3832 hybrid_clock.cc:648] HybridClock initialized: now 1786142585677693 us; error 66 us; skew 500 ppm
Aug 07 22:43:05 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:05.683734 3832 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:43:05.684625 3832 webserver.cc:460] Webserver started at http://127.26.189.254:36583/ using document root <none> and password file <none>
I20260807 22:43:05.685149 3832 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:05.685221 3832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:05.685428 3832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:05.687506 3832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "039cdd340ced43ec84ea0d40cbc865a7"
format_stamp: "Formatted at 2026-08-07 22:43:05 on dist-test-slave-4tdj"
server_key: "7bdc231e7e137f565fcfdb473b70cafe"
server_key_iv: "992f10a97283733b611a18de38af7d53"
server_key_version: "encryptionkey@0"
I20260807 22:43:05.688045 3832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "039cdd340ced43ec84ea0d40cbc865a7"
format_stamp: "Formatted at 2026-08-07 22:43:05 on dist-test-slave-4tdj"
server_key: "7bdc231e7e137f565fcfdb473b70cafe"
server_key_iv: "992f10a97283733b611a18de38af7d53"
server_key_version: "encryptionkey@0"
I20260807 22:43:05.691536 3832 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:05.693704 3862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:05.694891 3832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:05.694998 3832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "039cdd340ced43ec84ea0d40cbc865a7"
format_stamp: "Formatted at 2026-08-07 22:43:05 on dist-test-slave-4tdj"
server_key: "7bdc231e7e137f565fcfdb473b70cafe"
server_key_iv: "992f10a97283733b611a18de38af7d53"
server_key_version: "encryptionkey@0"
I20260807 22:43:05.695101 3832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:05.720834 3832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:05.723855 3832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:05.724030 3832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:05.730890 3832 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:36125
I20260807 22:43:05.730911 3937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:36125 every 8 connection(s)
I20260807 22:43:05.731796 3832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:05.734499 3939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:05.734807 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 3832
I20260807 22:43:05.734920 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:05.735188 27383 external_mini_cluster.cc:1428] Setting key 51f609345439557c75e5f16d115ae0d4
I20260807 22:43:05.741335 3939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7: Bootstrap starting.
I20260807 22:43:05.743714 3939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7: Neither blocks nor log segments found. Creating new log.
Aug 07 22:43:05 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:05.744521 3939 log.cc:824] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:05.746743 3939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7: No bootstrap required, opened a new log
I20260807 22:43:05.749368 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:05.736415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.26.189.254:36125)
0807 22:43:05.736837 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:05.736856 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:05.736899 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:43:05.737628 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:05.737652 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:05.738055 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:05.738447 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:05.739092 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:05.740544 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:05.741345 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:05.741999 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:05.742151 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:05.744590 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:05.744596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:05.744622 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:05.744653 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:05.746491 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:05.746897 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:05.746903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:05.746909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:05.747045 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:05.747282 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:05.747285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:05.747287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:05.747768 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:05.747945 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:05.748127 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 22:43:05.748409 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
I20260807 22:43:05.750985 3939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "039cdd340ced43ec84ea0d40cbc865a7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36125 } }
I20260807 22:43:05.751202 3939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:05.751257 3939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039cdd340ced43ec84ea0d40cbc865a7, State: Initialized, Role: FOLLOWER
I20260807 22:43:05.751668 3939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "039cdd340ced43ec84ea0d40cbc865a7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36125 } }
I20260807 22:43:05.751797 3939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:05.751860 3939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:05.751957 3939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:05.752749 3939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "039cdd340ced43ec84ea0d40cbc865a7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36125 } }
I20260807 22:43:05.753046 3939 leader_election.cc:302] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 039cdd340ced43ec84ea0d40cbc865a7; no voters:
I20260807 22:43:05.753312 3939 leader_election.cc:288] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:05.753415 3951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:05.753618 3951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [term 1 LEADER]: Becoming Leader. State: Replica: 039cdd340ced43ec84ea0d40cbc865a7, State: Running, Role: LEADER
I20260807 22:43:05.753903 3951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "039cdd340ced43ec84ea0d40cbc865a7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36125 } }
I20260807 22:43:05.754451 3939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:05.755486 3952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "039cdd340ced43ec84ea0d40cbc865a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "039cdd340ced43ec84ea0d40cbc865a7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36125 } } }
I20260807 22:43:05.755486 3954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 039cdd340ced43ec84ea0d40cbc865a7. Latest consensus state: current_term: 1 leader_uuid: "039cdd340ced43ec84ea0d40cbc865a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "039cdd340ced43ec84ea0d40cbc865a7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36125 } } }
I20260807 22:43:05.755646 3952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:05.755646 3954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:05.756126 3963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:05.759109 3963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:05.764276 3963 catalog_manager.cc:1347] Generated new cluster ID: f1730a5b5a8b4fe591b248efad810e3c
I20260807 22:43:05.764339 3963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:05.781306 3963 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:05.782398 3963 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:05.786355 3963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 039cdd340ced43ec84ea0d40cbc865a7: Generated new TSK 0
I20260807 22:43:05.786988 3963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:43:05.822417 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36125
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:05.935281 3989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:05.935519 3989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:05.935586 3989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:05.939522 3989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:05.939577 3989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:05.939649 3989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:05.944625 3989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:05.945663 3989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:05.946638 3989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:05.955750 3994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:05.955790 3997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:05.955750 4000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:05.956337 3989 server_base.cc:1034] running on GCE node
I20260807 22:43:05.956740 3989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:05.957298 3989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:05.958455 3989 hybrid_clock.cc:648] HybridClock initialized: now 1786142585958429 us; error 42 us; skew 500 ppm
Aug 07 22:43:05 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:05.961433 3989 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:43:05.962500 3989 webserver.cc:460] Webserver started at http://127.26.189.193:42961/ using document root <none> and password file <none>
I20260807 22:43:05.963173 3989 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:05.963258 3989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:05.963482 3989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:05.966166 3989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "2dd10823dd644964a93e119ccf3ccb0b"
format_stamp: "Formatted at 2026-08-07 22:43:05 on dist-test-slave-4tdj"
server_key: "83c33a169d70785bdb9a0c2659a0809b"
server_key_iv: "6c21d4231e4138b5c70cb56c863692c5"
server_key_version: "encryptionkey@0"
I20260807 22:43:05.966866 3989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "2dd10823dd644964a93e119ccf3ccb0b"
format_stamp: "Formatted at 2026-08-07 22:43:05 on dist-test-slave-4tdj"
server_key: "83c33a169d70785bdb9a0c2659a0809b"
server_key_iv: "6c21d4231e4138b5c70cb56c863692c5"
server_key_version: "encryptionkey@0"
I20260807 22:43:05.971474 3989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:43:05.974474 4011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:05.975919 3989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:05.976037 3989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "2dd10823dd644964a93e119ccf3ccb0b"
format_stamp: "Formatted at 2026-08-07 22:43:05 on dist-test-slave-4tdj"
server_key: "83c33a169d70785bdb9a0c2659a0809b"
server_key_iv: "6c21d4231e4138b5c70cb56c863692c5"
server_key_version: "encryptionkey@0"
I20260807 22:43:05.976156 3989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:06.007815 3989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:06.010856 3989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:06.011014 3989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:06.011574 3989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:06.012377 3989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:06.012422 3989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.012460 3989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:06.012480 3989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.021032 3989 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:35347
I20260807 22:43:06.021096 4184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:35347 every 8 connection(s)
I20260807 22:43:06.021873 3989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:06.026376 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 3989
I20260807 22:43:06.026508 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:06.026857 27383 external_mini_cluster.cc:1428] Setting key a9e9103cb75a5271f1b0260c738aaab1
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:06.035861 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.023623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:41253 (local address 127.26.189.254:36125)
0807 22:43:06.023749 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.023752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.024553 (+ 801us) server_negotiation.cc:407] Connection header received
0807 22:43:06.025867 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.025870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.025917 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.025991 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.027566 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.028099 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.029613 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.029953 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.030049 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.032439 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.032443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.032445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.032474 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.033905 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.034403 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.034407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.034409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.034491 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.034816 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.034818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.034819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.034994 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.035080 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.035559 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.035694 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:43:06.036621 4188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.023976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36125 (local address 127.26.189.193:41253)
0807 22:43:06.024409 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:06.024450 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:06.025682 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:06.026255 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:06.026265 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:06.026836 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:06.027411 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:06.027427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.028259 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.028262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:06.029484 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:06.029496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.030110 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.030114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:06.030346 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.030871 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:06.030887 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:06.032279 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:06.034015 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:06.034021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:06.034035 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:06.034322 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:06.034575 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:06.034578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:06.034580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:06.034717 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:06.035095 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:06.035101 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:06.035334 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:43:06.035485 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:43:06.035794 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":125,"threads_started":1}
I20260807 22:43:06.037763 4186 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36125
I20260807 22:43:06.037978 4186 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:06.038465 4186 heartbeater.cc:511] Master 127.26.189.254:36125 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:43:06.040035 3888 ts_manager.cc:194] Registered new tserver with Master: 2dd10823dd644964a93e119ccf3ccb0b (127.26.189.193:35347)
I20260807 22:43:06.041505 3888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:41253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:43:06.054453 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36125
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:06.161723 4191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:06.161959 4191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:06.162019 4191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:06.166173 4191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:06.166237 4191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:06.166337 4191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:06.171515 4191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:06.172631 4191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:06.173632 4191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:06.182900 4200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:06.182924 4199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:06.182946 4204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:06.183063 4191 server_base.cc:1034] running on GCE node
I20260807 22:43:06.183538 4191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:06.184127 4191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:06.185276 4191 hybrid_clock.cc:648] HybridClock initialized: now 1786142586185244 us; error 44 us; skew 500 ppm
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.187901 4191 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:43:06.188683 4191 webserver.cc:460] Webserver started at http://127.26.189.194:34855/ using document root <none> and password file <none>
I20260807 22:43:06.189181 4191 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:06.189239 4191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:06.189402 4191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:06.191464 4191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "e218f5744af7440db8969b91903d04ee"
format_stamp: "Formatted at 2026-08-07 22:43:06 on dist-test-slave-4tdj"
server_key: "b5a95a3e5660b71cf14b9c0182cce2a1"
server_key_iv: "f92edcbdd69b09b2ded694c9cefbf195"
server_key_version: "encryptionkey@0"
I20260807 22:43:06.191977 4191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "e218f5744af7440db8969b91903d04ee"
format_stamp: "Formatted at 2026-08-07 22:43:06 on dist-test-slave-4tdj"
server_key: "b5a95a3e5660b71cf14b9c0182cce2a1"
server_key_iv: "f92edcbdd69b09b2ded694c9cefbf195"
server_key_version: "encryptionkey@0"
I20260807 22:43:06.195590 4191 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:06.197881 4215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.199100 4191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:06.199206 4191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "e218f5744af7440db8969b91903d04ee"
format_stamp: "Formatted at 2026-08-07 22:43:06 on dist-test-slave-4tdj"
server_key: "b5a95a3e5660b71cf14b9c0182cce2a1"
server_key_iv: "f92edcbdd69b09b2ded694c9cefbf195"
server_key_version: "encryptionkey@0"
I20260807 22:43:06.199298 4191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:06.221095 4191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:06.223750 4191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:06.223919 4191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:06.224472 4191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:06.225287 4191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:06.225337 4191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.225387 4191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:06.225415 4191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.233850 4191 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:40827
I20260807 22:43:06.233886 4401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:40827 every 8 connection(s)
I20260807 22:43:06.234726 4191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:06.240206 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 4191
I20260807 22:43:06.240351 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:06.240638 27383 external_mini_cluster.cc:1428] Setting key 9f8370147c4a9d36db61b62ba8e6c88b
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:06.247844 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.236354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:34169 (local address 127.26.189.254:36125)
0807 22:43:06.236469 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.236473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.237257 (+ 784us) server_negotiation.cc:407] Connection header received
0807 22:43:06.238644 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.238647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.238707 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.238778 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.239879 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.240626 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.241753 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.241980 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.242071 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.244426 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.244429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.244431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.244453 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.245833 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.246326 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.246330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.246332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.246431 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.246754 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.246758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.246760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.246992 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.247095 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.247540 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.247674 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:43:06.248672 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.236693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36125 (local address 127.26.189.194:34169)
0807 22:43:06.237118 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:06.237159 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:06.238435 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:06.238949 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:06.238956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:06.239295 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:06.239740 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:06.239753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.240766 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.240769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:06.241623 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:06.241633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.242128 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.242132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:06.242317 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.242874 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:06.242889 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:06.244268 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:06.245942 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:06.245947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:06.245958 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:06.246213 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:06.246518 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:06.246520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:06.246522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:06.246659 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:06.247088 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:06.247095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:06.247311 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 22:43:06.247459 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:43:06.247715 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":106,"threads_started":1}
I20260807 22:43:06.249811 4402 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36125
I20260807 22:43:06.250034 4402 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:06.250530 4402 heartbeater.cc:511] Master 127.26.189.254:36125 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:43:06.251478 3888 ts_manager.cc:194] Registered new tserver with Master: e218f5744af7440db8969b91903d04ee (127.26.189.194:40827)
I20260807 22:43:06.252051 3888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:34169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:43:06.266906 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36125
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:06.375931 4409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:06.376175 4409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:06.376231 4409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:06.380288 4409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:06.380352 4409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:06.380462 4409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:06.385912 4409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:06.386917 4409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:06.387979 4409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:06.397353 4421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:06.397380 4423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:06.397367 4419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:06.397753 4409 server_base.cc:1034] running on GCE node
I20260807 22:43:06.398155 4409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:06.398799 4409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:06.399967 4409 hybrid_clock.cc:648] HybridClock initialized: now 1786142586399945 us; error 39 us; skew 500 ppm
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.402590 4409 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:43:06.403493 4409 webserver.cc:460] Webserver started at http://127.26.189.195:46307/ using document root <none> and password file <none>
I20260807 22:43:06.404024 4409 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:06.404091 4409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:06.404296 4409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:06.406332 4409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "13982b1b870648f1adbd0460132a77e0"
format_stamp: "Formatted at 2026-08-07 22:43:06 on dist-test-slave-4tdj"
server_key: "27e314abe42e216f409422cbe61d8014"
server_key_iv: "9c9e3f3a15ce137b5d41f7c64f2a4cf9"
server_key_version: "encryptionkey@0"
I20260807 22:43:06.406886 4409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "13982b1b870648f1adbd0460132a77e0"
format_stamp: "Formatted at 2026-08-07 22:43:06 on dist-test-slave-4tdj"
server_key: "27e314abe42e216f409422cbe61d8014"
server_key_iv: "9c9e3f3a15ce137b5d41f7c64f2a4cf9"
server_key_version: "encryptionkey@0"
I20260807 22:43:06.410521 4409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:06.412878 4430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.414016 4409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:06.414129 4409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "13982b1b870648f1adbd0460132a77e0"
format_stamp: "Formatted at 2026-08-07 22:43:06 on dist-test-slave-4tdj"
server_key: "27e314abe42e216f409422cbe61d8014"
server_key_iv: "9c9e3f3a15ce137b5d41f7c64f2a4cf9"
server_key_version: "encryptionkey@0"
I20260807 22:43:06.414220 4409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:06.446513 4409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:06.449498 4409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:06.449676 4409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:06.450270 4409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:06.451151 4409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:06.451198 4409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.451236 4409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:06.451256 4409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:06.459780 4409 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:38579
I20260807 22:43:06.459815 4613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:38579 every 8 connection(s)
I20260807 22:43:06.460623 4409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:06.463024 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 4409
I20260807 22:43:06.463161 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:06.463474 27383 external_mini_cluster.cc:1428] Setting key 0dc93e81ce040b456abe08e1cc37aa3e
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:06.473551 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.462282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:58573 (local address 127.26.189.254:36125)
0807 22:43:06.462451 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.462456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.463384 (+ 928us) server_negotiation.cc:407] Connection header received
0807 22:43:06.465005 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.465009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.465078 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.465163 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.466365 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.466891 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.467963 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.468201 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.468278 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.470570 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.470574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.470576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.470611 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.471860 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.472372 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.472375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.472377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.472463 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.472723 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.472725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.472726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.472885 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.472957 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.473346 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.473436 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:43:06.474282 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.462717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36125 (local address 127.26.189.195:58573)
0807 22:43:06.463219 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:06.463277 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:06.464792 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:06.465312 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:06.465318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:06.465759 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:06.466222 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:06.466236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.466984 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.466987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:06.467855 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:06.467864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.468310 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.468312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:06.468443 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.468954 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:06.468970 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:06.470413 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:06.471982 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:06.471986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:06.471997 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:06.472275 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:06.472529 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:06.472531 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:06.472532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:06.472653 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:06.472954 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:06.472958 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:06.473171 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 22:43:06.473304 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 22:43:06.473526 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":102,"threads_started":1}
I20260807 22:43:06.475350 4615 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36125
I20260807 22:43:06.475553 4615 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:06.476034 4615 heartbeater.cc:511] Master 127.26.189.254:36125 requested a full tablet report, sending...
I20260807 22:43:06.477149 3886 ts_manager.cc:194] Registered new tserver with Master: 13982b1b870648f1adbd0460132a77e0 (127.26.189.195:38579)
I20260807 22:43:06.477702 3886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:58573
I20260807 22:43:06.488545 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/test-admin.keytab.
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.503177 27383 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 22:43:06.505784 4625 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:06.504437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.26.189.193:35347)
0807 22:43:06.504764 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.504779 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.504799 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:43:06.504846 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.504851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.505020 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.505145 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.505614 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34556: BlockingRecv error: recv got EOF from 127.0.0.1:34556 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":109,"threads_started":1}
W20260807 22:43:06.505861 27383 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.506839 27383 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.507673 27383 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:43:06.515884 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.508407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.26.189.193:35347)
0807 22:43:06.508531 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.508534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.508582 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:43:06.508685 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.508687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.508723 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.508785 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.509215 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.509952 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.510757 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.511095 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.511183 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.513151 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.513159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.513171 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.513191 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.514493 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.514847 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.514851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.514852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.514921 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.515094 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.515096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.515097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.515312 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.515447 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.515597 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.515733 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 22:43:06.519371 4625 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:06.518428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.26.189.193:35347)
0807 22:43:06.518546 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.518550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.518638 (+ 88us) server_negotiation.cc:407] Connection header received
0807 22:43:06.518714 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.518716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.518776 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.518847 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.519281 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34580: BlockingRecv error: recv got EOF from 127.0.0.1:34580 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 22:43:06.519501 27383 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.520362 27383 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.521106 27383 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:43:06.529163 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.521826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.26.189.193:35347)
0807 22:43:06.521960 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.521963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.522039 (+ 76us) server_negotiation.cc:407] Connection header received
0807 22:43:06.522154 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.522157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.522193 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.522255 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.522779 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.523499 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.524353 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.524598 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.524690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.526724 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.526727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.526729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.526750 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.527836 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.528200 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.528203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.528205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.528282 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.528469 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.528471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.528472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.528615 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.528710 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.528963 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.529049 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 22:43:06.531452 4625 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:06.530715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.26.189.193:35347)
0807 22:43:06.530835 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.530839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.530917 (+ 78us) server_negotiation.cc:407] Connection header received
0807 22:43:06.530974 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.530977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.531013 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.531071 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.531365 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34596: BlockingRecv error: recv got EOF from 127.0.0.1:34596 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 22:43:06.531804 27383 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142556986508-27383-0/krb5kdc/krb5cc)
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.532651 27383 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:06.533366 27383 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:43:06.540235 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:06.534048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.26.189.193:35347)
0807 22:43:06.534173 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:06.534176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:06.534223 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:43:06.534345 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:06.534348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:06.534386 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:06.534443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:06.534839 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.535294 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.535976 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:06.536172 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:06.536233 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:06.538007 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:06.538009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:06.538011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:06.538031 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:06.539080 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.539426 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.539428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.539430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.539491 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:06.539663 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:06.539665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:06.539666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:06.539791 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:06.539872 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:06.540015 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 22:43:06.540090 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:43:06.541040 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 3989
I20260807 22:43:06.558736 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 4191
I20260807 22:43:06.567119 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 4409
I20260807 22:43:06.585753 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 3832
2026-08-07T22:43:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2284 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[4643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[4643](info): set up 2 sockets
Aug 07 22:43:06 dist-test-slave-4tdj krb5kdc[4643](info): commencing operation
krb5kdc: starting...
W20260807 22:43:07.725584 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:07 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab.
Entry for principal oryx/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab.
I20260807 22:43:07.845062 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39689
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab
--principal=oryx/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:07.955408 4663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:07.955672 4663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:07.955729 4663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:07.959631 4663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:07.959707 4663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:07.959733 4663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:07.959755 4663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:07.959782 4663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:07.964896 4663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab
--principal=oryx/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:07.965885 4663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:07.966921 4663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:07.975122 4670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:07.975224 4663 server_base.cc:1034] running on GCE node
W20260807 22:43:07.975124 4674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:07.975124 4669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:07.975955 4663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:07.977017 4663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:07.978173 4663 hybrid_clock.cc:648] HybridClock initialized: now 1786142587978157 us; error 31 us; skew 500 ppm
Aug 07 22:43:07 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:07.980846 4663 init.cc:377] Logged in from keytab as oryx/127.26.189.254@KRBTEST.COM (short username oryx)
I20260807 22:43:07.981663 4663 webserver.cc:460] Webserver started at http://127.26.189.254:34843/ using document root <none> and password file <none>
I20260807 22:43:07.982178 4663 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:07.982241 4663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:07.982471 4663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:07.984439 4663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "9146e72314e44334b85ed4be5c622d1d"
format_stamp: "Formatted at 2026-08-07 22:43:07 on dist-test-slave-4tdj"
server_key: "c705a791aa36f86f5c9c414724ad8b9d"
server_key_iv: "f6e26f21ffcbfccfd58192a8779125a4"
server_key_version: "encryptionkey@0"
I20260807 22:43:07.984982 4663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "9146e72314e44334b85ed4be5c622d1d"
format_stamp: "Formatted at 2026-08-07 22:43:07 on dist-test-slave-4tdj"
server_key: "c705a791aa36f86f5c9c414724ad8b9d"
server_key_iv: "f6e26f21ffcbfccfd58192a8779125a4"
server_key_version: "encryptionkey@0"
I20260807 22:43:07.988513 4663 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:07.990805 4686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:07.991923 4663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:07.992035 4663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "9146e72314e44334b85ed4be5c622d1d"
format_stamp: "Formatted at 2026-08-07 22:43:07 on dist-test-slave-4tdj"
server_key: "c705a791aa36f86f5c9c414724ad8b9d"
server_key_iv: "f6e26f21ffcbfccfd58192a8779125a4"
server_key_version: "encryptionkey@0"
I20260807 22:43:07.992121 4663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:08.036679 4663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:08.039407 4663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:08.039592 4663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:08.046177 4663 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:39689
I20260807 22:43:08.046455 4780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:39689 every 8 connection(s)
I20260807 22:43:08.047137 4663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:08.049779 4781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:08.051283 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 4663
I20260807 22:43:08.051406 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:08.051680 27383 external_mini_cluster.cc:1428] Setting key ed2f8dbb801cd24576b66b6d0e87a1b7
I20260807 22:43:08.056192 4781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Bootstrap starting.
I20260807 22:43:08.058791 4781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:08.059638 4781 log.cc:824] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:08.061589 4781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: No bootstrap required, opened a new log
I20260807 22:43:08.066092 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.052985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.26.189.254:39689)
0807 22:43:08.053415 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.053435 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.053477 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:43:08.054306 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.054332 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.054742 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.055180 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.055797 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.057223 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.058039 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.058752 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.058916 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.061283 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.061290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.061305 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.061335 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.063191 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.063590 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.063597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.063603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.063764 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.063986 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.063989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.063991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.064472 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.064664 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.064839 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.065126 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":115,"threads_started":1}
I20260807 22:43:08.066022 4781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } }
I20260807 22:43:08.066318 4781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:08.066380 4781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9146e72314e44334b85ed4be5c622d1d, State: Initialized, Role: FOLLOWER
I20260807 22:43:08.066833 4781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } }
I20260807 22:43:08.066987 4781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:08.067094 4781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:08.067207 4781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:08.068150 4781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } }
I20260807 22:43:08.068475 4781 leader_election.cc:302] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9146e72314e44334b85ed4be5c622d1d; no voters:
I20260807 22:43:08.068764 4781 leader_election.cc:288] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:08.068853 4788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:08.069077 4788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 LEADER]: Becoming Leader. State: Replica: 9146e72314e44334b85ed4be5c622d1d, State: Running, Role: LEADER
I20260807 22:43:08.069394 4788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } }
I20260807 22:43:08.069846 4781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:08.070875 4791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9146e72314e44334b85ed4be5c622d1d. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } }
I20260807 22:43:08.070995 4791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:08.071370 4798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:08.071400 4790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } }
I20260807 22:43:08.072063 4790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:08.074111 4798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:08.079303 4798 catalog_manager.cc:1347] Generated new cluster ID: b6ef1092b6ac4220b882fa7b5465e75d
I20260807 22:43:08.079365 4798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:08.109741 4798 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:08.111027 4798 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:08.125547 4798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Generated new TSK 0
I20260807 22:43:08.126228 4798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab.
Entry for principal oryx/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab.
I20260807 22:43:08.155171 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39689
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:08.264263 4814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:08.264508 4814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:08.264549 4814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:08.268512 4814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:08.268577 4814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:08.268666 4814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:08.274045 4814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:08.275041 4814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:08.275995 4814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:08.285399 4819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.285405 4821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.285405 4825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.285460 4814 server_base.cc:1034] running on GCE node
I20260807 22:43:08.285948 4814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:08.286612 4814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:08.287761 4814 hybrid_clock.cc:648] HybridClock initialized: now 1786142588287737 us; error 39 us; skew 500 ppm
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:08.290346 4814 init.cc:377] Logged in from keytab as oryx/127.26.189.193@KRBTEST.COM (short username oryx)
I20260807 22:43:08.291182 4814 webserver.cc:460] Webserver started at http://127.26.189.193:38125/ using document root <none> and password file <none>
I20260807 22:43:08.291697 4814 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:08.291754 4814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:08.291929 4814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:08.293947 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "da1b681065e3d1a66a26ed9c05ef36b2"
server_key_iv: "9b1d55ab674725ce9148acfd8e352145"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.294507 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "da1b681065e3d1a66a26ed9c05ef36b2"
server_key_iv: "9b1d55ab674725ce9148acfd8e352145"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.298131 4814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:43:08.300388 4835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.301642 4814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:08.301740 4814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "da1b681065e3d1a66a26ed9c05ef36b2"
server_key_iv: "9b1d55ab674725ce9148acfd8e352145"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.301837 4814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:08.312809 4814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:08.315328 4814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:08.315479 4814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:08.316000 4814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:08.316756 4814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:08.316805 4814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.316855 4814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:08.316882 4814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.325132 4814 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:33931
I20260807 22:43:08.325167 5035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:33931 every 8 connection(s)
I20260807 22:43:08.325985 4814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:08.331550 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 4814
I20260807 22:43:08.331665 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:08.331956 27383 external_mini_cluster.cc:1428] Setting key f031423a4fc9fb8c400cc7b62fc51c98
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:08.339233 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.327670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:44565 (local address 127.26.189.254:39689)
0807 22:43:08.327829 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.327835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.328615 (+ 780us) server_negotiation.cc:407] Connection header received
0807 22:43:08.329891 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.329894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.329942 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.330013 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.331164 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.331731 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.333013 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.333313 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.333415 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.335771 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.335775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.335777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.335805 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.337209 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.337669 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.337672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.337674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.337774 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.338071 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.338074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.338076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.338344 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.338446 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.338924 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.339053 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:43:08.339951 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.328030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:44565)
0807 22:43:08.328472 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:08.328513 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:08.329713 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:08.330156 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:08.330162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:08.330554 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:08.331040 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.331054 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.331851 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.331854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.332889 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.332901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.333455 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.333459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.333618 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.334159 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:08.334175 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:08.335598 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:08.337314 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.337319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.337331 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.337576 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.337861 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.337863 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.337865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.337984 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.338442 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:08.338447 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:08.338685 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 22:43:08.338839 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 22:43:08.339082 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":124,"threads_started":1}
I20260807 22:43:08.341132 5037 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39689
I20260807 22:43:08.341389 5037 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:08.341962 5037 heartbeater.cc:511] Master 127.26.189.254:39689 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:43:08.343580 4717 ts_manager.cc:194] Registered new tserver with Master: 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931)
I20260807 22:43:08.344734 4717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:44565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab.
Entry for principal oryx/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab.
I20260807 22:43:08.359460 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39689
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:08.467388 5047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:08.467660 5047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:08.467718 5047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:08.471776 5047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:08.471833 5047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:08.471911 5047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:08.477029 5047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:08.478085 5047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:08.479133 5047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:08.488425 5052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.488417 5053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.488590 5047 server_base.cc:1034] running on GCE node
W20260807 22:43:08.488452 5056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.489096 5047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:08.489727 5047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:08.490869 5047 hybrid_clock.cc:648] HybridClock initialized: now 1786142588490842 us; error 40 us; skew 500 ppm
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:08.493393 5047 init.cc:377] Logged in from keytab as oryx/127.26.189.194@KRBTEST.COM (short username oryx)
I20260807 22:43:08.494256 5047 webserver.cc:460] Webserver started at http://127.26.189.194:35975/ using document root <none> and password file <none>
I20260807 22:43:08.494819 5047 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:08.494879 5047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:08.495074 5047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:08.497200 5047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "67bc7776e4884366969cd99d4ef7648f"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "97167971c2d6330e772f5837cc028545"
server_key_iv: "01f2200762699f029a3a6b66f5ee2d9d"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.497741 5047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "67bc7776e4884366969cd99d4ef7648f"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "97167971c2d6330e772f5837cc028545"
server_key_iv: "01f2200762699f029a3a6b66f5ee2d9d"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.501361 5047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:43:08.503674 5070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.504897 5047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:08.505014 5047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "67bc7776e4884366969cd99d4ef7648f"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "97167971c2d6330e772f5837cc028545"
server_key_iv: "01f2200762699f029a3a6b66f5ee2d9d"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.505110 5047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:08.526646 5047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:08.529368 5047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:08.529523 5047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:08.530078 5047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:08.530920 5047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:08.530966 5047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.531006 5047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:08.531028 5047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.539258 5047 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:42137
I20260807 22:43:08.539287 5258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:42137 every 8 connection(s)
I20260807 22:43:08.540108 5047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:08.545133 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 5047
I20260807 22:43:08.545245 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:08.545507 27383 external_mini_cluster.cc:1428] Setting key bd3c535be8fc19245d05721de628af6f
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:08.553602 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.541855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53599 (local address 127.26.189.254:39689)
0807 22:43:08.542005 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.542010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.542870 (+ 860us) server_negotiation.cc:407] Connection header received
0807 22:43:08.544165 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.544168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.544214 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.544285 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.545478 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.546120 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.547504 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.547729 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.547811 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.550393 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.550396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.550399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.550425 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.551618 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.552113 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.552117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.552118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.552205 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.552530 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.552534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.552535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.552720 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.552805 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.553307 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.553434 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:43:08.554472 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.542231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:53599)
0807 22:43:08.542721 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:08.542766 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:08.543980 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:08.544433 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:08.544440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:08.544807 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:08.545323 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.545340 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.546256 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.546259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.547376 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.547389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.547879 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.547883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.548082 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.548676 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:08.548696 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:08.550210 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:08.551731 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.551736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.551752 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.552004 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.552287 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.552290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.552292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.552430 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.552819 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:08.552826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:08.553065 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 22:43:08.553229 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 22:43:08.553561 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:43:08.555626 5259 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39689
I20260807 22:43:08.555867 5259 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:08.556354 5259 heartbeater.cc:511] Master 127.26.189.254:39689 requested a full tablet report, sending...
I20260807 22:43:08.557426 4717 ts_manager.cc:194] Registered new tserver with Master: 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.195@KRBTEST.COM" created.
I20260807 22:43:08.558079 4717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:53599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab.
Entry for principal oryx/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab.
I20260807 22:43:08.572961 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39689
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:08.681226 5272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:08.681452 5272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:08.681492 5272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:08.685516 5272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:08.685572 5272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:08.685632 5272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:08.690735 5272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:08.691676 5272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:08.692596 5272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:08.701681 5277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.701776 5279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.701678 5281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.701956 5272 server_base.cc:1034] running on GCE node
I20260807 22:43:08.702481 5272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:08.703109 5272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:08.704272 5272 hybrid_clock.cc:648] HybridClock initialized: now 1786142588704255 us; error 27 us; skew 500 ppm
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:08.707038 5272 init.cc:377] Logged in from keytab as oryx/127.26.189.195@KRBTEST.COM (short username oryx)
I20260807 22:43:08.707918 5272 webserver.cc:460] Webserver started at http://127.26.189.195:39805/ using document root <none> and password file <none>
I20260807 22:43:08.708422 5272 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:08.708479 5272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:08.708660 5272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:08.710726 5272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "8b02e13866fe4e898143821c82626f11"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "339c3ad00f6390cf21f60abfacecbf1c"
server_key_iv: "ece36a7a26774fed9ee9a5e6938ffd08"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.711241 5272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "8b02e13866fe4e898143821c82626f11"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "339c3ad00f6390cf21f60abfacecbf1c"
server_key_iv: "ece36a7a26774fed9ee9a5e6938ffd08"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.714776 5272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:08.717072 5295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.718237 5272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:08.718376 5272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "8b02e13866fe4e898143821c82626f11"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "339c3ad00f6390cf21f60abfacecbf1c"
server_key_iv: "ece36a7a26774fed9ee9a5e6938ffd08"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.718477 5272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:08.739650 5272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:08.742353 5272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:08.742525 5272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:08.743121 5272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:08.743927 5272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:08.743973 5272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.744019 5272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:08.744046 5272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.752467 5272 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:33201
I20260807 22:43:08.752506 5470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:33201 every 8 connection(s)
I20260807 22:43:08.753315 5272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:08.759060 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 5272
I20260807 22:43:08.759179 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:08.759481 27383 external_mini_cluster.cc:1428] Setting key 19b610fa2549bae50bdc209586c69536
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:08.766261 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.755059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:56265 (local address 127.26.189.254:39689)
0807 22:43:08.755202 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.755206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.755983 (+ 777us) server_negotiation.cc:407] Connection header received
0807 22:43:08.757284 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.757287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.757335 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.757406 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.758541 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.759088 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.760549 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.760796 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.760905 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.763369 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.763372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.763374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.763398 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.764586 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.765057 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.765059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.765060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.765139 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.765405 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.765408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.765409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.765571 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.765644 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.766043 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.766145 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:43:08.767102 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.755414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:56265)
0807 22:43:08.755836 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:08.755877 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:08.757084 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:08.757553 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:08.757560 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:08.757912 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:08.758405 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.758420 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.759229 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.759233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.760415 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.760426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.760970 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.760975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.761145 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.761708 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:08.761727 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:08.763213 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:08.764689 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.764693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.764704 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.764977 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.765200 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.765202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.765211 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.765338 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.765642 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:08.765647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:08.765857 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 22:43:08.766007 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:43:08.766250 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260807 22:43:08.768132 5472 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39689
I20260807 22:43:08.768352 5472 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:08.768811 5472 heartbeater.cc:511] Master 127.26.189.254:39689 requested a full tablet report, sending...
I20260807 22:43:08.769734 4717 ts_manager.cc:194] Registered new tserver with Master: 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201)
I20260807 22:43:08.770351 4717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:56265
I20260807 22:43:08.773301 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.189.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.253.keytab.
Entry for principal oryx/127.26.189.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.253.keytab.
I20260807 22:43:08.799351 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:43053
--webserver_interface=127.26.189.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.253.keytab
--principal=oryx/127.26.189.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39689,127.26.189.253:43053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:08.909035 5482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:08.909351 5482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:08.909412 5482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:08.913555 5482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:08.913609 5482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:08.913626 5482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:08.913643 5482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:08.913671 5482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:08.918927 5482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39689,127.26.189.253:43053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.253.keytab
--principal=oryx/127.26.189.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:43053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.189.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
W20260807 22:43:08.919111 5482 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.189.254:39689,127.26.189.253:43053'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 22:43:08.919943 5482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:08.920918 5482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:08.929345 5495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:08.929471 5493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.929574 5482 server_base.cc:1034] running on GCE node
W20260807 22:43:08.929706 5498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:08.930212 5482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:08.931268 5482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:08.932451 5482 hybrid_clock.cc:648] HybridClock initialized: now 1786142588932417 us; error 52 us; skew 500 ppm
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:08.935201 5482 init.cc:377] Logged in from keytab as oryx/127.26.189.253@KRBTEST.COM (short username oryx)
I20260807 22:43:08.936072 5482 webserver.cc:460] Webserver started at http://127.26.189.253:44167/ using document root <none> and password file <none>
I20260807 22:43:08.936594 5482 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:08.936661 5482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:08.936862 5482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:08.938890 5482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/instance:
uuid: "600910733a9044d4b25b30003f4a2f86"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "260ff1491528d7761ef334d194d1b174"
server_key_iv: "a45e2e28e17b6e51f1be6b51c816bff3"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.939534 5482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal/instance:
uuid: "600910733a9044d4b25b30003f4a2f86"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "260ff1491528d7761ef334d194d1b174"
server_key_iv: "a45e2e28e17b6e51f1be6b51c816bff3"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.943207 5482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:08.945490 5511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:08.946758 5482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:08.946882 5482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
uuid: "600910733a9044d4b25b30003f4a2f86"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "260ff1491528d7761ef334d194d1b174"
server_key_iv: "a45e2e28e17b6e51f1be6b51c816bff3"
server_key_version: "encryptionkey@0"
I20260807 22:43:08.946970 5482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:08.962037 5482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:08.964967 5482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:08.965131 5482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:08.971755 5482 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.253:43053
I20260807 22:43:08.971994 5580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.253:43053 every 8 connection(s)
I20260807 22:43:08.972657 5482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
I20260807 22:43:08.975404 5581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:08.975836 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 5482
I20260807 22:43:08.980051 5581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.253@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:08.990728 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.977493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.26.189.253:43053)
0807 22:43:08.977919 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.977933 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.977968 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:43:08.978705 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.978723 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.979024 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.979315 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.980093 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.981093 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.982182 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.982870 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.983033 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.985598 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.985603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.985612 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.985638 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.987666 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.988133 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.988139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.988145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.988282 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.988562 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.988565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.988566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.988997 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.989137 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.989546 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.989796 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"spinlock_wait_cycles":1408,"thread_start_us":137,"threads_started":1}
I20260807 22:43:08.991030 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.981698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.26.189.254:39689)
0807 22:43:08.981818 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.981822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.982094 (+ 272us) server_negotiation.cc:407] Connection header received
0807 22:43:08.982285 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.982307 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.982366 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.982430 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.983211 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.983932 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.984998 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.985273 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.985365 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.987860 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.987865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.987867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.987896 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.989310 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.989802 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.989805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.989807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.989881 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.990142 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.990144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.990145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.990380 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.990469 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.990776 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.990891 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:43:08.991087 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.981618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39268)
0807 22:43:08.981959 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:08.982003 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:08.982131 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:08.982490 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:08.982495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:08.982909 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:08.983083 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.983090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.984076 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.984078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.984870 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.984882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.985415 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.985420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.985514 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.985960 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:08.985975 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:08.987732 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:08.989413 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.989419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.989424 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.989715 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.989958 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.989961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.989963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.990065 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.990484 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:08.990490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:08.990601 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 22:43:08.990743 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 22:43:08.990905 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"spinlock_wait_cycles":3840,"thread_start_us":127,"threads_started":1}
I20260807 22:43:08.992244 5581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:08 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142588, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.253@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
I20260807 22:43:08.999712 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.992665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55810)
0807 22:43:08.992747 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:08.992757 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:08.992814 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:08.992994 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:08.992997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:08.993136 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:08.993531 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.993537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.994200 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.994203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.994731 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:08.994739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.995138 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.995141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:08.995251 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.995759 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:08.995776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:08.996934 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:08.998614 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.998617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.998618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.998786 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.999017 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:08.999019 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:08.999021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:08.999080 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:08.999364 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:08.999366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:08.999434 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 22:43:08.999500 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:43:08.999591 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":2}
I20260807 22:43:08.999789 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:08.992736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.26.189.253:43053)
0807 22:43:08.992827 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:08.992831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:08.992844 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:08.992876 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:08.992879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:08.992915 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:08.992994 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:08.993629 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.994096 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.994828 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:08.995050 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:08.995131 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:08.997057 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:08.997061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:08.997063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:08.997084 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:08.998505 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.998862 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.998865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.998866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.998939 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:08.999148 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:08.999150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:08.999151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:08.999299 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:08.999375 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:08.999537 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:43:08.999643 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":35}
I20260807 22:43:09.002228 5581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Bootstrap starting.
I20260807 22:43:09.004366 5581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:09.005072 5581 log.cc:824] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:09.006928 5581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: No bootstrap required, opened a new log
I20260807 22:43:09.009809 5581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } }
I20260807 22:43:09.010001 5581 raft_consensus.cc:374] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:09.010066 5581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600910733a9044d4b25b30003f4a2f86, State: Initialized, Role: FOLLOWER
I20260807 22:43:09.010481 5581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } }
I20260807 22:43:09.010885 5589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } } }
I20260807 22:43:09.010987 5589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:09.011539 5581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:43:09.015728 5602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:43:09.015795 5602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:43:09.017544 5482 master_runner.cc:428] Detected that this master 600910733a9044d4b25b30003f4a2f86 is joining an existing cluster
I20260807 22:43:09.017589 5482 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.189.253:43053
I20260807 22:43:09.029630 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.019493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39278 (local address 127.26.189.254:39689)
0807 22:43:09.019664 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.019670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.020286 (+ 616us) server_negotiation.cc:407] Connection header received
0807 22:43:09.020465 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.020468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.020552 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.020668 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.021299 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.022543 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.023765 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.024098 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.024182 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.025658 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.025663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.025666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.025704 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.027549 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.028062 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.028066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.028068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.028160 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.028528 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.028532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.028534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.028769 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.028850 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.029334 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.029475 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:43:09.029639 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.019452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39278)
0807 22:43:09.020196 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.020211 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.020319 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.020741 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.020746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.021001 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.021159 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.021169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.022692 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.022696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.023605 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.023617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.024365 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.024369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.024495 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.025006 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.025023 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.025486 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.027681 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.027684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.027687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.027937 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.028282 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.028285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.028287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.028393 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.028936 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.028939 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.029059 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.029254 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.029417 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"spinlock_wait_cycles":1792,"thread_start_us":96,"threads_started":1}
I20260807 22:43:09.031689 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.019939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.26.189.253:43053)
0807 22:43:09.020060 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.020095 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.023780 (+ 3685us) server_negotiation.cc:407] Connection header received
0807 22:43:09.023920 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.023923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.023960 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.024001 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.024602 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.025272 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.026391 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.026704 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.026780 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.028464 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.028467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.028470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.028491 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.030078 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.030602 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.030605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.030607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.030682 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.030926 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.030929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.030931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.031134 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.031213 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.031445 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.031568 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":30}
I20260807 22:43:09.031693 5610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.019861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55822)
0807 22:43:09.023698 (+ 3837us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.023711 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.023834 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.024110 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.024112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.024330 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.024468 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.024476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.025408 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.025411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.026232 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.026244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.026920 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.026923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.027014 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.027467 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.027481 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.028337 (+ 856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.030170 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.030174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.030175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.030490 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.030762 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.030765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.030767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.030842 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.031220 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.031223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.031300 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.031374 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.031504 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3770,"mutex_wait_us":3,"spinlock_wait_cycles":2816,"thread_start_us":94,"threads_started":1}
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.254@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
I20260807 22:43:09.042645 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.034569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.26.189.253:43053)
0807 22:43:09.034664 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.034668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.035030 (+ 362us) server_negotiation.cc:407] Connection header received
0807 22:43:09.035194 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.035197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.035242 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.035302 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.035884 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.036340 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.037165 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.037374 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.037455 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.039837 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.039840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.039842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.039862 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.041063 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.041496 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.041499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.041501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.041572 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.041815 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.041818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.041820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.041975 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.042057 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.042401 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.042503 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:43:09.042688 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.034576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55836)
0807 22:43:09.034909 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.034949 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.035073 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.035347 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.035353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.035680 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.035777 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.035783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.036435 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.036439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.037058 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.037067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.037468 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.037472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.037588 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.038060 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.038072 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.039707 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.041176 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.041186 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.041191 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.041404 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.041654 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.041658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.041660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.041738 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.042074 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.042081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.042188 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.042348 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.042526 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
I20260807 22:43:09.044507 4717 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.189.253:43053: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "600910733a9044d4b25b30003f4a2f86"
member_type: NON_VOTER
last_known_addr {
host: "127.26.189.253"
port: 43053
}
attrs {
promote: true
}
}
dest_uuid: "9146e72314e44334b85ed4be5c622d1d"
cas_config_opid_index: -1
I20260807 22:43:09.045169 4717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } }
I20260807 22:43:09.046964 5614 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 600910733a9044d4b25b30003f4a2f86 (127.26.189.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } }
I20260807 22:43:09.047806 5545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:09.048810 5614 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.189.253:43053
I20260807 22:43:09.049156 5545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Refusing update from remote peer 9146e72314e44334b85ed4be5c622d1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 22:43:09.049199 5482 master.cc:537] Master@127.26.189.253:43053 shutting down...
I20260807 22:43:09.049571 5614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } }
I20260807 22:43:09.049664 5614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:09.049784 5615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:43:09.051575 4690 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 22:43:09.052471 4690 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d -> Peer 600910733a9044d4b25b30003f4a2f86 (127.26.189.253:43053): Couldn't send request to peer 600910733a9044d4b25b30003f4a2f86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:09.056164 5482 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:09.056413 5482 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:09.056515 5482 tablet_replica.cc:330] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: stopping tablet replica
I20260807 22:43:09.061559 5482 master.cc:552] Master@127.26.189.253:43053 shutdown complete.
I20260807 22:43:09.061604 5482 master_runner.cc:305] Clearing existing system tablet
I20260807 22:43:09.061815 5482 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.061865 5482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.062942 5482 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:43:09.063685 5621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.063880 5482 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.064008 5482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:09.064045 5482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
uuid: "600910733a9044d4b25b30003f4a2f86"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "260ff1491528d7761ef334d194d1b174"
server_key_iv: "a45e2e28e17b6e51f1be6b51c816bff3"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.065709 5482 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:09.067677 5482 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:43:09.067760 5482 log.cc:1196] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 22:43:09.067974 5482 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Deleting consensus metadata
I20260807 22:43:09.068136 5482 master_runner.cc:315] Copying system tablet from 127.26.189.254:39689
I20260807 22:43:09.069146 5482 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: tablet copy: Beginning tablet copy session from remote peer at address 127.26.189.254:39689
WARNING: no policy specified for oryx/127.26.189.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.252@KRBTEST.COM" created.
I20260807 22:43:09.077229 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.069733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.26.189.254:39689)
0807 22:43:09.069876 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.069881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.070276 (+ 395us) server_negotiation.cc:407] Connection header received
0807 22:43:09.070484 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.070487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.070545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.070657 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.071212 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.071722 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.072610 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.072852 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.072941 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.074314 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.074319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.074321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.074346 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.075683 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.076116 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.076119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.076121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.076200 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.076465 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.076468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.076469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.076635 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.076719 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.076966 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.077066 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:43:09.077739 5628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.069635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39282)
0807 22:43:09.070173 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.070191 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.070348 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.070726 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.070729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.070965 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.071103 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.071112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.071851 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.071855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.072492 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.072501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.073072 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.073076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.073182 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.073743 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.073757 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.074142 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.075785 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.075788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.075791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.076001 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.076299 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.076301 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.076302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.076376 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.076726 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.076729 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.076818 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.076903 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.077038 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":87,"threads_started":1}
I20260807 22:43:09.078271 4753 tablet_copy_service.cc:140] P 9146e72314e44334b85ed4be5c622d1d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 600910733a9044d4b25b30003f4a2f86 ({username='oryx', principal='oryx/127.26.189.253@KRBTEST.COM'} at 127.0.0.1:39282)
I20260807 22:43:09.078492 4753 tablet_copy_service.cc:161] P 9146e72314e44334b85ed4be5c622d1d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 600910733a9044d4b25b30003f4a2f86 at {username='oryx', principal='oryx/127.26.189.253@KRBTEST.COM'} at 127.0.0.1:39282: session id = 600910733a9044d4b25b30003f4a2f86-00000000000000000000000000000000
I20260807 22:43:09.079825 4753 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:43:09.081189 5482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:43:09.083194 5482 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: tablet copy: Starting download of 0 data blocks...
I20260807 22:43:09.083355 5482 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: tablet copy: Starting download of 1 WAL segments...
I20260807 22:43:09.084856 5482 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:43:09.086508 4753 tablet_copy_service.cc:342] P 9146e72314e44334b85ed4be5c622d1d: Request end of tablet copy session 600910733a9044d4b25b30003f4a2f86-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.189.253@KRBTEST.COM'} at 127.0.0.1:39282
I20260807 22:43:09.086609 4753 tablet_copy_service.cc:433] P 9146e72314e44334b85ed4be5c622d1d: ending tablet copy session 600910733a9044d4b25b30003f4a2f86-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 600910733a9044d4b25b30003f4a2f86
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.252.keytab.
Entry for principal oryx/127.26.189.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.252.keytab.
I20260807 22:43:09.093039 5482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:43:09.093366 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:43697
--webserver_interface=127.26.189.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.252.keytab
--principal=oryx/127.26.189.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:09.094504 5630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.095142 5631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.095613 5634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.096274 5482 server_base.cc:1034] running on GCE node
I20260807 22:43:09.096498 5482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.096828 5482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.104123 5482 hybrid_clock.cc:648] HybridClock initialized: now 1786142589104111 us; error 32 us; skew 500 ppm
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.105495 5482 init.cc:377] Logged in from keytab as oryx/127.26.189.253@KRBTEST.COM (short username oryx)
I20260807 22:43:09.105978 5482 webserver.cc:460] Webserver started at http://127.26.189.253:35627/ using document root <none> and password file <none>
I20260807 22:43:09.106158 5482 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.106195 5482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.107334 5482 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:09.108251 5649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.108501 5482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:09.108574 5482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
uuid: "600910733a9044d4b25b30003f4a2f86"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "260ff1491528d7761ef334d194d1b174"
server_key_iv: "a45e2e28e17b6e51f1be6b51c816bff3"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.108665 5482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.116245 5482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:09.118069 5482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:09.122617 5482 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.253:43053
I20260807 22:43:09.122643 5746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.253:43053 every 8 connection(s)
I20260807 22:43:09.123122 5482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
I20260807 22:43:09.124497 5747 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:43:09.125392 5747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Bootstrap starting.
I20260807 22:43:09.133358 5747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:43:09.133643 5747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Bootstrap complete.
I20260807 22:43:09.134083 5747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } }
I20260807 22:43:09.134209 5747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 600910733a9044d4b25b30003f4a2f86, State: Initialized, Role: LEARNER
I20260807 22:43:09.134325 5747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } }
I20260807 22:43:09.134663 5750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } }
I20260807 22:43:09.134757 5747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:09.134840 5750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:09.138072 5766 catalog_manager.cc:1259] Loaded cluster ID: b6ef1092b6ac4220b882fa7b5465e75d
I20260807 22:43:09.138176 5766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: loading cluster ID for follower catalog manager: success
I20260807 22:43:09.139778 5766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: acquiring CA information for follower catalog manager: success
I20260807 22:43:09.140790 5766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 22:43:09.243151 5640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:09.243398 5640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:09.243461 5640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:09.247488 5640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:09.247555 5640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:09.247575 5640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:09.247601 5640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:09.247632 5640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:09.252801 5640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.252.keytab
--principal=oryx/127.26.189.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:43697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.189.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:09.253815 5640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:09.254840 5640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:09.263108 5772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.263144 5640 server_base.cc:1034] running on GCE node
W20260807 22:43:09.263131 5773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.263108 5775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.263840 5640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.264910 5640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.266091 5640 hybrid_clock.cc:648] HybridClock initialized: now 1786142589266068 us; error 34 us; skew 500 ppm
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.268980 5640 init.cc:377] Logged in from keytab as oryx/127.26.189.252@KRBTEST.COM (short username oryx)
I20260807 22:43:09.269919 5640 webserver.cc:460] Webserver started at http://127.26.189.252:33937/ using document root <none> and password file <none>
I20260807 22:43:09.270573 5640 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.270650 5640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.270857 5640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:09.272943 5640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/instance:
uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-4tdj"
server_key: "e287ebf32e727294666345a34e1cc1db"
server_key_iv: "f936d0a23a3ec26ef7a4a91a4e49ffdd"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.273488 5640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal/instance:
uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-4tdj"
server_key: "e287ebf32e727294666345a34e1cc1db"
server_key_iv: "f936d0a23a3ec26ef7a4a91a4e49ffdd"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.277071 5640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 22:43:09.279443 5786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.280612 5640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:09.280715 5640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-4tdj"
server_key: "e287ebf32e727294666345a34e1cc1db"
server_key_iv: "f936d0a23a3ec26ef7a4a91a4e49ffdd"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.280815 5640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.295001 5640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:09.297748 5640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:09.297912 5640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:09.304721 5640 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.252:43697
I20260807 22:43:09.304967 5878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.252:43697 every 8 connection(s)
I20260807 22:43:09.305701 5640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
I20260807 22:43:09.308457 5879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:09.311481 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 5640
I20260807 22:43:09.312491 5879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.252@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.189.252@KRBTEST.COM
I20260807 22:43:09.324211 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.314323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.26.189.254:39689)
0807 22:43:09.314453 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.314457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.314653 (+ 196us) server_negotiation.cc:407] Connection header received
0807 22:43:09.314797 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.314800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.314847 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.314933 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.315849 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.316368 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.317456 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.317695 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.317804 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.320393 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.320397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.320400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.320429 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.321931 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.322608 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.322612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.322613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.322697 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.323057 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.323060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.323062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.323280 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.323376 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.323914 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.324043 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:43:09.325215 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.314085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39298)
0807 22:43:09.314505 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.314551 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.314687 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.314992 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.314997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.315390 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.315700 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.315713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.316489 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.316492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.317331 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.317341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.317835 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.317839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.317998 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.318775 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.318792 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.320249 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.322124 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.322131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.322141 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.322479 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.322786 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.322789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.322791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.322937 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.323379 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.323384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.323681 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.323840 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.324166 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":5,"thread_start_us":127,"threads_started":1}
I20260807 22:43:09.325215 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.312978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.26.189.252:43697)
0807 22:43:09.313382 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.313397 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.313434 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:43:09.314131 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.314154 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.314605 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.314893 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.315703 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.316674 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.317491 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.317927 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.318019 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.320913 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.320918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.320929 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.320951 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.322728 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.323123 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.323128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.323129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.323198 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.323453 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.323455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.323457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.323742 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.323926 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.324125 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.324246 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260807 22:43:09.326661 5879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.252@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
I20260807 22:43:09.334425 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.327218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55838)
0807 22:43:09.327560 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.327575 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.327657 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.327878 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.327881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.328043 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.328142 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.328148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.329037 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.329040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.329585 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.329594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.330025 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.330029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.330137 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.330535 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.330545 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.331700 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.333232 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.333235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.333236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.333414 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.333633 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.333635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.333636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.333700 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.334005 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.334007 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.334089 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.334176 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.334274 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270}
I20260807 22:43:09.334484 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.327326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55838 (local address 127.26.189.253:43053)
0807 22:43:09.327571 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.327576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.327618 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:43:09.327731 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.327734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.327782 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.327855 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.328256 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.328938 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.329693 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.329931 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.330016 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.331823 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.331826 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.331828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.331854 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.333131 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.333488 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.333490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.333491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.333558 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.333756 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.333758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.333759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.333920 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.333990 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.334226 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.334360 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
I20260807 22:43:09.335186 5879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.252@KRBTEST.COM for oryx/127.26.189.252@KRBTEST.COM
I20260807 22:43:09.341866 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.335640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34750)
0807 22:43:09.335744 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.335757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.335820 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.336033 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.336035 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.336163 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.336233 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.336238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.336949 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.336952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.337444 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.337452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.337835 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.337837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.337890 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.338257 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.338266 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.339382 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.340794 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.340796 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.340798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.340963 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.341152 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.341154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.341156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.341217 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.341481 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.341483 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.341549 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.341625 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.341726 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 22:43:09.341897 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.335726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.26.189.252:43697)
0807 22:43:09.335838 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.335842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.335854 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:09.335886 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.335888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.335951 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.336015 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.336340 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.336837 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.337527 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.337760 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.337829 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.339493 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.339496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.339498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.339519 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.340713 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.341028 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.341030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.341032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.341095 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.341273 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.341275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.341276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.341421 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.341487 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.341667 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.341778 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":32}
I20260807 22:43:09.345165 5879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Bootstrap starting.
I20260807 22:43:09.346980 5037 heartbeater.cc:503] Master 127.26.189.254:39689 was elected leader, sending a full tablet report...
I20260807 22:43:09.347558 5879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:09.348263 5879 log.cc:824] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:09.350109 5879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: No bootstrap required, opened a new log
I20260807 22:43:09.353029 5879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } }
I20260807 22:43:09.353236 5879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:09.353292 5879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7e3a5bf79b4799b2d2836f1df5bc52, State: Initialized, Role: FOLLOWER
I20260807 22:43:09.353717 5879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } }
I20260807 22:43:09.354154 5887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } } }
I20260807 22:43:09.354267 5887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:09.354882 5879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:43:09.360118 5904 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:43:09.360198 5904 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:43:09.362720 5640 master_runner.cc:428] Detected that this master 1e7e3a5bf79b4799b2d2836f1df5bc52 is joining an existing cluster
I20260807 22:43:09.362784 5640 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.189.252:43697
I20260807 22:43:09.373945 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.365144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.26.189.252:43697)
0807 22:43:09.365266 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.365271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.365984 (+ 713us) server_negotiation.cc:407] Connection header received
0807 22:43:09.366079 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.366082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.366115 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.366158 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.367040 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.367707 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369519 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.369812 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369870 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.371188 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.371192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.371194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.371217 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.372505 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.372873 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.372876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.372878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.372951 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.373179 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.373182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.373184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.373340 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.373440 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.373647 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.373789 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":40}
I20260807 22:43:09.373960 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.365054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34762)
0807 22:43:09.365902 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.365912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.365997 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.366272 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.366274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.366826 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.366931 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.366939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.367833 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.367836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.369422 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.369433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.369981 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.370145 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.370719 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.370734 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.371067 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.372606 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.372609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.372611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.372792 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.373024 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.373027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.373029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.373107 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.373422 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.373425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.373505 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.373601 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.373735 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":141,"spinlock_wait_cycles":3456,"thread_start_us":77,"threads_started":1}
I20260807 22:43:09.378464 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 4814
I20260807 22:43:09.393779 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.365514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55854 (local address 127.26.189.253:43053)
0807 22:43:09.365757 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.365762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.366582 (+ 820us) server_negotiation.cc:407] Connection header received
0807 22:43:09.366681 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.366684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.366738 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.366810 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.367401 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.368212 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369618 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.369973 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.370051 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.390197 (+ 20146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.390203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.390206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.390236 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.391876 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.392352 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.392356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.392358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.392441 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.392735 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.392739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.392741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.392970 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.393036 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.393458 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.393608 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260807 22:43:09.394378 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.365426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55854)
0807 22:43:09.366468 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.366480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.366582 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.366937 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.366940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.367135 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.367260 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.367268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.368356 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.368359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.369469 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.369481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.370208 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.370212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.370365 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.389493 (+ 19128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.389511 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.389980 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.391994 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.391997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.392000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.392241 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.392549 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.392552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.392554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.392638 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.393119 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.393122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.393267 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.393998 (+ 731us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.394167 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":173,"thread_start_us":71,"threads_started":1}
I20260807 22:43:09.412642 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.365732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.26.189.254:39689)
0807 22:43:09.365872 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.365877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.365892 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:43:09.365935 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.365938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.365986 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.366069 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.366667 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.367496 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369402 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.369724 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369794 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.399924 (+ 30130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.399935 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.399940 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.399984 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.410676 (+ 10692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.411259 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.411264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.411266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.411379 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.411692 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.411696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.411698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.411944 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.412044 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.412342 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.412479 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:43:09.412642 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.364994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39314)
0807 22:43:09.365415 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.365429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.365517 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.366085 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.366095 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.366387 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.366532 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.366541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.367625 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.367628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.368537 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.368549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.369964 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.369967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.370142 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.390922 (+ 20780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.390937 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.391584 (+ 647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.410855 (+ 19271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.410859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.410862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.411135 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.411491 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.411494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.411496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.411595 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.412053 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.412056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.412176 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.412282 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.412438 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":63,"spinlock_wait_cycles":1792,"thread_start_us":97,"threads_started":1}
I20260807 22:43:09.416956 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:33931
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:39689
--webserver_port=38125
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142587, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.254@KRBTEST.COM for oryx/127.26.189.252@KRBTEST.COM
I20260807 22:43:09.426709 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.417137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.26.189.252:43697)
0807 22:43:09.417649 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.417654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.417666 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:09.417736 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.417739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.417789 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.417863 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.418410 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.418995 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.420114 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.420417 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.420534 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.423167 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.423171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.423174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.423203 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.424811 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.425329 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.425333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.425335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.425427 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.425715 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.425718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.425719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.425941 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.426044 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.426400 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.426522 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425}
I20260807 22:43:09.426712 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.417049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34768)
0807 22:43:09.417273 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.417287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.417364 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.417911 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.417915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.418127 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.418238 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.418245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.419107 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.419111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.419973 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.419984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.420530 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.420534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.420632 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.421112 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.421143 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.423003 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.424947 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.424951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.424953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.425227 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.425526 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.425529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.425531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.425624 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.426044 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.426047 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.426157 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.426253 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.426544 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 22:43:09.428359 4717 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.189.252:43697: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
member_type: NON_VOTER
last_known_addr {
host: "127.26.189.252"
port: 43697
}
attrs {
promote: true
}
}
dest_uuid: "9146e72314e44334b85ed4be5c622d1d"
cas_config_opid_index: 5
I20260807 22:43:09.429072 4717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } }
I20260807 22:43:09.430564 5615 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.432003 5615 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.189.252:43697
I20260807 22:43:09.432179 5615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.432269 5615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:09.433192 5837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:09.433359 5640 master.cc:537] Master@127.26.189.252:43697 shutting down...
I20260807 22:43:09.434695 5837 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Refusing update from remote peer 9146e72314e44334b85ed4be5c622d1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 22:43:09.435439 5614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:43:09.437425 4689 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d -> Peer 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252:43697): Couldn't send request to peer 1e7e3a5bf79b4799b2d2836f1df5bc52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:09.439909 5640 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:09.440212 5640 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:09.440327 5640 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: stopping tablet replica
I20260807 22:43:09.440618 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.430975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.26.189.253:43053)
0807 22:43:09.431129 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.431134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.431338 (+ 204us) server_negotiation.cc:407] Connection header received
0807 22:43:09.431508 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.431511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.431574 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.431642 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:09.432013 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.432862 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.438460 (+ 5598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.439146 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.439637 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.439795 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.440355 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.440485 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:43:09.441146 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.430723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55870)
0807 22:43:09.431270 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.431284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.431400 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.431794 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.431797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.431812 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:09.431895 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.431901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.436095 (+ 4194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.436099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.437417 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.437429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.439901 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.439904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.440079 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.440170 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.440929 (+ 759us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.441017 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474}
I20260807 22:43:09.441772 5704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 LEARNER]: Refusing update from remote peer 9146e72314e44334b85ed4be5c622d1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 22:43:09.442345 5614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 22:43:09.444775 5640 master.cc:552] Master@127.26.189.252:43697 shutdown complete.
I20260807 22:43:09.444813 5640 master_runner.cc:305] Clearing existing system tablet
I20260807 22:43:09.445046 5640 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.445093 5640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.444901 5704 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.446264 5640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:43:09.447161 5750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9146e72314e44334b85ed4be5c622d1d. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.447274 5750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:09.447192 5929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: NON_VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: true } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.447320 5929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:09.448743 5614 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: attempting to promote NON_VOTER 600910733a9044d4b25b30003f4a2f86 to VOTER
I20260807 22:43:09.449191 5614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } }
I20260807 22:43:09.450392 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.450081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34784)
0807 22:43:09.450202 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.450280 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 22:43:09.450532 5704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 LEARNER]: Refusing update from remote peer 9146e72314e44334b85ed4be5c622d1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 22:43:09.451007 5617 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 22:43:09.451395 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.451093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34796)
0807 22:43:09.451283 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.451333 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260807 22:43:09.451537 4689 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d -> Peer 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252:43697): Couldn't send request to peer 1e7e3a5bf79b4799b2d2836f1df5bc52. Status: Network error: Client connection negotiation failed: client connection to 127.26.189.252:43697: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:09.452494 5615 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 600910733a9044d4b25b30003f4a2f86 (127.26.189.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.453876 5615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.453980 5615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:09.454094 5615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.454205 5615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:09.454416 5927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.454741 5704 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 600910733a9044d4b25b30003f4a2f86 (127.26.189.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.456142 5929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.456295 5929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:09.456400 5640 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.456537 5640 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260807 22:43:09.456586 5640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-4tdj"
server_key: "e287ebf32e727294666345a34e1cc1db"
server_key_iv: "f936d0a23a3ec26ef7a4a91a4e49ffdd"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.458630 5640 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:09.460920 5640 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:43:09.461010 5640 log.cc:1196] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 22:43:09.461270 5640 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Deleting consensus metadata
I20260807 22:43:09.461428 5640 master_runner.cc:315] Copying system tablet from 127.26.189.254:39689
I20260807 22:43:09.462618 5640 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: tablet copy: Beginning tablet copy session from remote peer at address 127.26.189.254:39689
I20260807 22:43:09.471534 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.463086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39316)
0807 22:43:09.463340 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.463591 (+ 251us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.463730 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.464013 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.464016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.464241 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.464345 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.464352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.465402 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.465405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.466284 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.466320 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.466859 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.466863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.466963 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.467490 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.467505 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.467909 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.469829 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.469832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.469834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.470076 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.470374 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.470377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.470379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.470474 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.470976 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.470979 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.471073 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.471192 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.471349 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":96,"threads_started":1}
I20260807 22:43:09.471544 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.463452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.26.189.254:39689)
0807 22:43:09.463561 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.463566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.463658 (+ 92us) server_negotiation.cc:407] Connection header received
0807 22:43:09.463827 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.463831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.463906 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.463989 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.464465 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.465289 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.466413 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.466747 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.466851 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.468055 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.468059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.468061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.468089 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.469719 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.470182 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.470185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.470187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.470276 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.470617 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.470621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.470623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.470836 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.470905 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.471275 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.471394 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:43:09.472344 4753 tablet_copy_service.cc:140] P 9146e72314e44334b85ed4be5c622d1d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1e7e3a5bf79b4799b2d2836f1df5bc52 ({username='oryx', principal='oryx/127.26.189.252@KRBTEST.COM'} at 127.0.0.1:39316)
I20260807 22:43:09.472436 4753 tablet_copy_service.cc:161] P 9146e72314e44334b85ed4be5c622d1d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1e7e3a5bf79b4799b2d2836f1df5bc52 at {username='oryx', principal='oryx/127.26.189.252@KRBTEST.COM'} at 127.0.0.1:39316: session id = 1e7e3a5bf79b4799b2d2836f1df5bc52-00000000000000000000000000000000
I20260807 22:43:09.473538 4753 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:43:09.474982 5640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:43:09.477262 5640 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: tablet copy: Starting download of 0 data blocks...
I20260807 22:43:09.477427 5640 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: tablet copy: Starting download of 1 WAL segments...
I20260807 22:43:09.478958 5640 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:43:09.480744 4753 tablet_copy_service.cc:342] P 9146e72314e44334b85ed4be5c622d1d: Request end of tablet copy session 1e7e3a5bf79b4799b2d2836f1df5bc52-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.189.252@KRBTEST.COM'} at 127.0.0.1:39316
I20260807 22:43:09.480824 4753 tablet_copy_service.cc:433] P 9146e72314e44334b85ed4be5c622d1d: ending tablet copy session 1e7e3a5bf79b4799b2d2836f1df5bc52-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1e7e3a5bf79b4799b2d2836f1df5bc52
I20260807 22:43:09.488292 5640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:09.489972 5938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.490252 5941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.490401 5640 server_base.cc:1034] running on GCE node
W20260807 22:43:09.490643 5939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.490916 5640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.491212 5640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.492344 5640 hybrid_clock.cc:648] HybridClock initialized: now 1786142589492331 us; error 32 us; skew 500 ppm
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.493976 5640 init.cc:377] Logged in from keytab as oryx/127.26.189.252@KRBTEST.COM (short username oryx)
I20260807 22:43:09.494463 5640 webserver.cc:460] Webserver started at http://127.26.189.252:46651/ using document root <none> and password file <none>
I20260807 22:43:09.494643 5640 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.494692 5640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.496061 5640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:09.497087 5952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.497367 5640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:09.497439 5640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
format_stamp: "Formatted at 2026-08-07 22:43:09 on dist-test-slave-4tdj"
server_key: "e287ebf32e727294666345a34e1cc1db"
server_key_iv: "f936d0a23a3ec26ef7a4a91a4e49ffdd"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.497514 5640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.526682 5640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:09.528681 5640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:09.533506 5640 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.252:43697
I20260807 22:43:09.534178 6029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.252:43697 every 8 connection(s)
I20260807 22:43:09.534348 5640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
I20260807 22:43:09.535861 6030 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:43:09.536649 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Bootstrap starting.
I20260807 22:43:09.546015 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:43:09.546268 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: Bootstrap complete.
I20260807 22:43:09.546780 6030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } }
I20260807 22:43:09.546900 6030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1e7e3a5bf79b4799b2d2836f1df5bc52, State: Initialized, Role: LEARNER
I20260807 22:43:09.546998 6030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } }
I20260807 22:43:09.547397 6030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:09.547334 6033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:09.547479 6033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:43:09.550729 6049 catalog_manager.cc:1259] Loaded cluster ID: b6ef1092b6ac4220b882fa7b5465e75d
I20260807 22:43:09.550781 6049 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: loading cluster ID for follower catalog manager: success
I20260807 22:43:09.552261 6049 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: acquiring CA information for follower catalog manager: success
I20260807 22:43:09.553313 6049 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:43:09.560050 5259 heartbeater.cc:503] Master 127.26.189.254:39689 was elected leader, sending a full tablet report...
W20260807 22:43:09.560901 5922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:09.561240 5922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:09.561318 5922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:09.565328 5922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:09.565380 5922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:09.565450 5922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:09.570657 5922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:33931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=38125
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:09.571710 5922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:09.572693 5922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:09.582019 6056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.582026 6057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.582080 6059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.582252 5922 server_base.cc:1034] running on GCE node
I20260807 22:43:09.582726 5922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.583324 5922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.584482 5922 hybrid_clock.cc:648] HybridClock initialized: now 1786142589584467 us; error 26 us; skew 500 ppm
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.587100 5922 init.cc:377] Logged in from keytab as oryx/127.26.189.193@KRBTEST.COM (short username oryx)
I20260807 22:43:09.587886 5922 webserver.cc:460] Webserver started at http://127.26.189.193:38125/ using document root <none> and password file <none>
I20260807 22:43:09.588393 5922 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.588476 5922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.592170 5922 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:43:09.594067 6068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.595268 5922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:09.595389 5922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "da1b681065e3d1a66a26ed9c05ef36b2"
server_key_iv: "9b1d55ab674725ce9148acfd8e352145"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.595937 5922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.610100 5922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:09.612951 5922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:09.613129 5922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:09.613703 5922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:09.614585 5922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:09.614641 5922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.614696 5922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:09.614724 5922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.623292 5922 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:33931
I20260807 22:43:09.623350 6239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:33931 every 8 connection(s)
I20260807 22:43:09.624228 5922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:09.631104 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 5922
I20260807 22:43:09.631292 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 5047
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.252@KRBTEST.COM
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:09.653491 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:42137
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:39689
--webserver_port=35975
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:09.656719 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.627496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:41547 (local address 127.26.189.253:43053)
0807 22:43:09.627632 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.627637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.627655 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:43:09.628054 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.628058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.628115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.628169 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.629529 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.630348 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648896 (+ 18548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.649252 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.649338 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.652562 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.652566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.652569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.652598 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.654328 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.654919 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.654923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.654925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.655020 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655429 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.655432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.655434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.655671 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.655740 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.656428 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.656551 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:43:09.657029 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.625879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:39401 (local address 127.26.189.254:39689)
0807 22:43:09.626010 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.626014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.627307 (+ 1293us) server_negotiation.cc:407] Connection header received
0807 22:43:09.628038 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.628041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.628101 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.628178 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.629505 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.630355 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648242 (+ 17887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.648583 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648733 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.652904 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.652908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.652911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.652939 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.655000 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655516 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.655520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.655522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.655636 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.656039 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.656042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.656044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.656256 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.656336 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.656744 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.656869 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:43:09.657034 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.625879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37221 (local address 127.26.189.252:43697)
0807 22:43:09.626179 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.626196 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.626975 (+ 779us) server_negotiation.cc:407] Connection header received
0807 22:43:09.628055 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.628058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.628119 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.628173 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.630363 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.631282 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648438 (+ 17156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.648733 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648822 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.653135 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.653139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.653140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.653169 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.654904 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655516 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.655520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.655522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.655740 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.656037 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.656041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.656042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.656256 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.656336 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.656744 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.656870 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":81,"threads_started":1}
I20260807 22:43:09.658782 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.627344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.26.189.193:41547)
0807 22:43:09.627594 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.627607 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.627836 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.628364 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.628370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.628823 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.629368 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.629378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.630522 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.630525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.648053 (+ 17528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.648071 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.649512 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.649515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.649599 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.650046 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.650066 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.652373 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.654452 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.654457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.654473 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.654804 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.655134 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.655138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.655324 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.655822 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.655829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.656143 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.657360 (+ 1217us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.657745 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":320,"spinlock_wait_cycles":27776,"thread_start_us":79,"threads_started":1}
I20260807 22:43:09.658774 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.626275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:39401)
0807 22:43:09.627209 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.627220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.627833 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.628362 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.628368 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.628815 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.629354 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.629374 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.630522 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.630525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.648094 (+ 17569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.648109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.648841 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.649114 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.650065 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.650086 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.652665 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.655103 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.655109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.655399 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.655841 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.655845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.655941 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.656396 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.656399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.656536 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.657241 (+ 705us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.657647 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":16158,"spinlock_wait_cycles":17920,"thread_start_us":668,"threads_started":1}
I20260807 22:43:09.658776 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.626275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.26.189.193:37221)
0807 22:43:09.626787 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.626843 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.627828 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.628364 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.628370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.628814 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.629371 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.629378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.631382 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.631385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.648312 (+ 16927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.648319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.648981 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.648984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.649116 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.649867 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.649890 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.652953 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.655031 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.655036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.655399 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.655823 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.655826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.655828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.655927 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.656396 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.656399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.656537 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.657241 (+ 704us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.657814 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":16282,"spinlock_wait_cycles":25344,"thread_start_us":110,"threads_started":1}
I20260807 22:43:09.660014 6241 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39689
I20260807 22:43:09.660226 6240 heartbeater.cc:347] Connected to a master server at 127.26.189.252:43697
I20260807 22:43:09.660256 6241 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.660239 6242 heartbeater.cc:347] Connected to a master server at 127.26.189.253:43053
I20260807 22:43:09.660332 6240 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.660378 6242 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.660862 6242 heartbeater.cc:511] Master 127.26.189.253:43053 requested a full tablet report, sending...
I20260807 22:43:09.660876 6240 heartbeater.cc:511] Master 127.26.189.252:43697 requested a full tablet report, sending...
I20260807 22:43:09.660863 6241 heartbeater.cc:511] Master 127.26.189.254:39689 requested a full tablet report, sending...
I20260807 22:43:09.662365 4717 ts_manager.cc:194] Re-registered known tserver with Master: 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931)
I20260807 22:43:09.662518 5972 ts_manager.cc:194] Registered new tserver with Master: 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931)
I20260807 22:43:09.663010 4717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:39401
I20260807 22:43:09.663682 5688 ts_manager.cc:194] Registered new tserver with Master: 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931)
W20260807 22:43:09.764398 6250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:09.764624 6250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:09.764664 6250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:09.768540 6250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:09.768587 6250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:09.768651 6250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:09.772331 5472 heartbeater.cc:503] Master 127.26.189.254:39689 was elected leader, sending a full tablet report...
I20260807 22:43:09.773869 6250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:42137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=35975
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:09.774864 6250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:09.775755 6250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:09.784844 6258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.784833 6256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.784901 6250 server_base.cc:1034] running on GCE node
W20260807 22:43:09.784842 6261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.785482 6250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.786026 6250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.787165 6250 hybrid_clock.cc:648] HybridClock initialized: now 1786142589787131 us; error 47 us; skew 500 ppm
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.789788 6250 init.cc:377] Logged in from keytab as oryx/127.26.189.194@KRBTEST.COM (short username oryx)
I20260807 22:43:09.790647 6250 webserver.cc:460] Webserver started at http://127.26.189.194:35975/ using document root <none> and password file <none>
I20260807 22:43:09.791154 6250 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.791234 6250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.794802 6250 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:09.796739 6269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.797909 6250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:09.798025 6250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "67bc7776e4884366969cd99d4ef7648f"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "97167971c2d6330e772f5837cc028545"
server_key_iv: "01f2200762699f029a3a6b66f5ee2d9d"
server_key_version: "encryptionkey@0"
I20260807 22:43:09.798565 6250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:09.816692 6250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:09.819165 6250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:09.819324 6250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:09.819851 6250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:09.820665 6250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:09.820711 6250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.820761 6250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:09.820791 6250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:09.829288 6250 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:42137
I20260807 22:43:09.829319 6448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:42137 every 8 connection(s)
I20260807 22:43:09.830224 6250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.252@KRBTEST.COM
I20260807 22:43:09.840478 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 6250
I20260807 22:43:09.840653 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 5272
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
I20260807 22:43:09.844009 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.833548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53853 (local address 127.26.189.254:39689)
0807 22:43:09.833675 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.833679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.833692 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:09.833861 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.833864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.833917 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.833987 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.835470 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.836036 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837440 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.837765 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837846 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.840468 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.840472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.840475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.840503 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.841896 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.842406 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.842410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.842411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.842483 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.842804 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.842806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.842808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.842974 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.843032 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.843727 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.843850 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:43:09.844781 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.831889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:48933 (local address 127.26.189.252:43697)
0807 22:43:09.832016 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.832021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.833194 (+ 1173us) server_negotiation.cc:407] Connection header received
0807 22:43:09.833971 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.833974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.834032 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.834109 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.836191 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.837015 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837867 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.838211 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.838313 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.841299 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.841304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.841307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.841340 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.843173 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.843563 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.843565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.843567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.843659 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.843959 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.843962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.843964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.844175 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.844267 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.844517 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.844646 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:43:09.845160 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.833245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:53853)
0807 22:43:09.833622 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.833635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.833724 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.834169 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.834177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.834700 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.835285 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.835305 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.836177 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.836180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.837292 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.837302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.837958 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.838072 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.838709 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.838728 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.840300 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.842004 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.842008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.842020 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.842281 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.842566 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.842568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.842570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.842713 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.843085 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.843090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.843421 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.843672 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.844046 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":98,"thread_start_us":70,"threads_started":1}
I20260807 22:43:09.845347 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.832286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.26.189.194:48933)
0807 22:43:09.833090 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.833103 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.833679 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.834244 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.834247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.834705 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.835318 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.835326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.837128 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.837741 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.837749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.838460 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.838463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.838525 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.838860 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.838873 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.841119 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.843285 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.843300 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.843301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.843471 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.843767 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.843769 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.843771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.843857 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.844271 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.844275 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.844353 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.844449 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.844576 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":106,"spinlock_wait_cycles":10240,"thread_start_us":125,"threads_started":1}
I20260807 22:43:09.846343 6449 heartbeater.cc:347] Connected to a master server at 127.26.189.252:43697
I20260807 22:43:09.846540 6449 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.846958 6449 heartbeater.cc:511] Master 127.26.189.252:43697 requested a full tablet report, sending...
I20260807 22:43:09.847959 5972 ts_manager.cc:194] Registered new tserver with Master: 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137)
I20260807 22:43:09.855724 6450 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39689
I20260807 22:43:09.855831 6450 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.856029 6450 heartbeater.cc:511] Master 127.26.189.254:39689 requested a full tablet report, sending...
I20260807 22:43:09.856689 4717 ts_manager.cc:194] Re-registered known tserver with Master: 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137)
I20260807 22:43:09.857303 4717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:53853
I20260807 22:43:09.858764 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.832274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.26.189.194:32891)
0807 22:43:09.832781 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:09.832831 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:09.833669 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:09.834171 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:09.834180 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:09.834744 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:09.835306 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.835315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.836174 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.836178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.837296 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:09.837305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.837873 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:09.838077 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.838742 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:09.838757 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:09.855168 (+ 16411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:09.857128 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.857132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.857134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.857366 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.857695 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:09.857698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:09.857700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:09.857798 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:09.858218 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:09.858221 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:09.858326 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 22:43:09.858439 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:43:09.858584 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":173,"spinlock_wait_cycles":11392,"thread_start_us":560,"threads_started":1}
I20260807 22:43:09.858778 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:09.831868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:32891 (local address 127.26.189.253:43053)
0807 22:43:09.832002 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:09.832006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:09.832950 (+ 944us) server_negotiation.cc:407] Connection header received
0807 22:43:09.833910 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:09.833913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:09.833971 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:09.834024 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:09.835441 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.836036 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837398 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:09.837732 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:09.837833 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:09.855329 (+ 17496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:09.855334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:09.855337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:09.855366 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:09.857003 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.857512 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.857515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.857517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.857609 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:09.857887 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:09.857890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:09.857891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:09.858110 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:09.858208 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:09.858505 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 22:43:09.858629 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:43:09.859448 6451 heartbeater.cc:347] Connected to a master server at 127.26.189.253:43053
I20260807 22:43:09.859532 6451 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:09.859709 6451 heartbeater.cc:511] Master 127.26.189.253:43053 requested a full tablet report, sending...
I20260807 22:43:09.860368 5688 ts_manager.cc:194] Registered new tserver with Master: 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137)
I20260807 22:43:09.861258 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:33201
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:39689
--webserver_port=39805
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:09.969465 6456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:09.969725 6456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:09.969767 6456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:09.973747 6456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:09.973799 6456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:09.973852 6456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:09.978976 6456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:33201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=39805
--tserver_master_addrs=127.26.189.254:39689,127.26.189.253:43053,127.26.189.252:43697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:09.979882 6456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:09.980782 6456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:09.989748 6463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:09.989749 6462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.989913 6456 server_base.cc:1034] running on GCE node
W20260807 22:43:09.989792 6466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:09.990410 6456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:09.990998 6456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:09.992142 6456 hybrid_clock.cc:648] HybridClock initialized: now 1786142589992121 us; error 39 us; skew 500 ppm
Aug 07 22:43:09 dist-test-slave-4tdj krb5kdc[4643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:09.994671 6456 init.cc:377] Logged in from keytab as oryx/127.26.189.195@KRBTEST.COM (short username oryx)
I20260807 22:43:09.995455 6456 webserver.cc:460] Webserver started at http://127.26.189.195:39805/ using document root <none> and password file <none>
I20260807 22:43:09.995949 6456 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:09.996028 6456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:09.999655 6456 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:10.001693 6479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.002967 6456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:10.003093 6456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "8b02e13866fe4e898143821c82626f11"
format_stamp: "Formatted at 2026-08-07 22:43:08 on dist-test-slave-4tdj"
server_key: "339c3ad00f6390cf21f60abfacecbf1c"
server_key_iv: "ece36a7a26774fed9ee9a5e6938ffd08"
server_key_version: "encryptionkey@0"
I20260807 22:43:10.003647 6456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:10.016652 6456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:10.019848 6456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:10.020056 6456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:10.020646 6456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:10.021521 6456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:10.021575 6456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.021632 6456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:10.021663 6456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:10.030915 6456 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:33201
I20260807 22:43:10.030994 6641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:33201 every 8 connection(s)
I20260807 22:43:10.031900 6456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:10.037067 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 6456
I20260807 22:43:10.040287 6655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.035563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.26.189.252:43697)
0807 22:43:10.035792 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.035806 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.035994 (+ 188us) server_negotiation.cc:407] Connection header received
0807 22:43:10.036211 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.036214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.036289 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.036406 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:10.036770 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.037431 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.038839 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.039465 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.039555 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.039721 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.040064 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.040145 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260807 22:43:10.040654 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.035104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34798)
0807 22:43:10.035926 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.035939 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.036080 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.036481 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.036484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.036505 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:10.036616 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.036623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.037553 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.037557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.038690 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.038702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.039822 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.039826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.039923 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.039948 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.040419 (+ 471us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.040512 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":71,"threads_started":1}
I20260807 22:43:10.041411 5987 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 22:43:10.041930 6033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9146e72314e44334b85ed4be5c622d1d. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:10.042065 6033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: This master's current role is: LEARNER
Aug 07 22:43:10 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.252@KRBTEST.COM
Aug 07 22:43:10 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:10.051213 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.033682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:48165 (local address 127.26.189.252:43697)
0807 22:43:10.033847 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.033853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.034911 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 22:43:10.036233 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.036236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.036287 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.036358 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.039886 (+ 3528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.040472 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.041992 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.042327 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.042417 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.046164 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.046168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.046171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.046197 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.047984 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.048633 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.048636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.048638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.048728 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.049330 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.049334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.049335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.049528 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.049611 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.050932 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.051046 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":896}
Aug 07 22:43:10 dist-test-slave-4tdj krb5kdc[4643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142589, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.253@KRBTEST.COM
I20260807 22:43:10.051541 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.034114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.26.189.195:48165)
0807 22:43:10.034738 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.034797 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.036018 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.036572 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.036581 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.038635 (+ 2054us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:10.039324 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.039345 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.040601 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.040605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.041861 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.041872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.042570 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.042574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.042826 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.043378 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:10.043397 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:10.045945 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:10.048152 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.048158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.048175 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.048516 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.048823 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.048826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.048828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.049224 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.049663 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:10.049669 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:10.049947 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.050140 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.050476 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":38,"spinlock_wait_cycles":1280,"thread_start_us":664,"threads_started":1}
I20260807 22:43:10.052695 6642 heartbeater.cc:347] Connected to a master server at 127.26.189.252:43697
I20260807 22:43:10.052914 6642 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:10.053431 6642 heartbeater.cc:511] Master 127.26.189.252:43697 requested a full tablet report, sending...
I20260807 22:43:10.054534 5972 ts_manager.cc:194] Registered new tserver with Master: 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201)
I20260807 22:43:10.055924 6652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.035201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.26.189.195:36439)
0807 22:43:10.035500 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.035512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.036009 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.036582 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.036586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.038618 (+ 2032us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:10.039557 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.039563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.042420 (+ 2857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.042426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.044189 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.044201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.044949 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.044953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.045047 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.045902 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:10.045918 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:10.051838 (+ 5920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:10.054404 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.054407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.054409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.054636 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.054920 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.054923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.054925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.054998 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.055412 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:10.055416 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:10.055507 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.055605 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.055730 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":763,"thread_start_us":114,"threads_started":1}
I20260807 22:43:10.056282 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.033834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:40379 (local address 127.26.189.254:39689)
0807 22:43:10.033955 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.033959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.035248 (+ 1289us) server_negotiation.cc:407] Connection header received
0807 22:43:10.036830 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.036833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.036879 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.036926 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.041235 (+ 4309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.041920 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.043088 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.043373 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.043437 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.046792 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.046795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.046797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.046821 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.052806 (+ 5985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.054215 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.054220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.054221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.054331 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.054887 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.054891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.054892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.055101 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.055177 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.055748 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.055861 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:43:10.056427 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.035563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:36439 (local address 127.26.189.253:43053)
0807 22:43:10.035759 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.035763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.035777 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:43:10.036234 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.036237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.036287 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.036338 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.039935 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.040689 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.044328 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.044679 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.044777 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.052490 (+ 7713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.052494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.052496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.052531 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.054266 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.054747 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.054750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.054752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.054828 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.055116 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.055119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.055121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.055321 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.055405 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.056171 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.056282 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 22:43:10.056993 6645 heartbeater.cc:347] Connected to a master server at 127.26.189.253:43053
I20260807 22:43:10.057117 6645 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:10.057312 6645 heartbeater.cc:511] Master 127.26.189.253:43053 requested a full tablet report, sending...
I20260807 22:43:10.057521 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.039727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.26.189.254:39689)
0807 22:43:10.041318 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.041331 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.041341 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:43:10.041368 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.041371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.041409 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.041483 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.042467 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.043183 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.044474 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.044777 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.044867 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.046396 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.046400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.046402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.046467 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.055496 (+ 9029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.055949 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.055952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.055954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.056054 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.056345 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.056349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.056351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.056559 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.056676 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.057245 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.057366 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1529,"thread_start_us":67,"threads_started":1}
I20260807 22:43:10.057799 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.034114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:40379)
0807 22:43:10.035162 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.035175 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.036721 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.037056 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.037059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.041010 (+ 3951us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:10.041119 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.041125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.042057 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.042060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.042966 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.042977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.043582 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.043585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.043682 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.044137 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:10.044151 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:10.046643 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:10.053285 (+ 6642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.053288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.053290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.053797 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.054690 (+ 893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:10.054693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:10.054695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:10.054780 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:10.055440 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:10.055443 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:10.055539 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.056775 (+ 1236us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.056929 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":4,"spinlock_wait_cycles":20480,"thread_start_us":206,"threads_started":1}
I20260807 22:43:10.058028 5688 ts_manager.cc:194] Registered new tserver with Master: 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201)
I20260807 22:43:10.058537 6643 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39689
I20260807 22:43:10.058624 6643 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:10.058698 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.039532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.26.189.253:43053)
0807 22:43:10.045060 (+ 5528us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.045065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.045648 (+ 583us) server_negotiation.cc:407] Connection header received
0807 22:43:10.045823 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.045827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.045877 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.045940 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.047889 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.048753 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.049849 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.050175 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.050254 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.051836 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.051840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.051842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.051866 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.055312 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.057167 (+ 1855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.057170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.057171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.057259 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.057556 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.057559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.057561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.057774 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.057885 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.058425 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.058535 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5458,"thread_start_us":91,"threads_started":1}
I20260807 22:43:10.058795 6643 heartbeater.cc:511] Master 127.26.189.254:39689 requested a full tablet report, sending...
I20260807 22:43:10.059469 4717 ts_manager.cc:194] Re-registered known tserver with Master: 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201)
I20260807 22:43:10.060127 4717 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:40379
I20260807 22:43:10.062889 4717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:43:10.066453 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.038970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.26.189.252:43697)
0807 22:43:10.039530 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.039534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.045360 (+ 5826us) server_negotiation.cc:407] Connection header received
0807 22:43:10.046794 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.046798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.046845 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.046904 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.048289 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.049014 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.050077 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.050419 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.050488 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.063346 (+ 12858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.063349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.063353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.063381 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.064919 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.065300 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.065304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.065306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.065393 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.065609 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.065612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.065614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.065842 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.065970 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.066145 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.066270 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":64,"threads_started":1}
W20260807 22:43:10.066779 4717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:10.069172 5614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: NON_VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: true } } }
I20260807 22:43:10.069291 5614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:10.072412 5614 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: attempting to promote NON_VOTER 1e7e3a5bf79b4799b2d2836f1df5bc52 to VOTER
I20260807 22:43:10.072742 5614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } }
I20260807 22:43:10.073781 5704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Refusing update from remote peer 9146e72314e44334b85ed4be5c622d1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 22:43:10.074110 6670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 22:43:10.075569 5986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 LEARNER]: Refusing update from remote peer 9146e72314e44334b85ed4be5c622d1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 22:43:10.075845 6671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.075990 6669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 22:43:10.076126 5704 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.077571 5986 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.078121 6669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.078224 6669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:10.078450 6671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.078544 6671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:10.079233 6033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.079346 6033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:10.079874 6670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.079983 6670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:10.080524 6682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9146e72314e44334b85ed4be5c622d1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:10.080631 6682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:10.087105 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.081796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:33931 (local address 127.0.0.1:36894)
0807 22:43:10.081968 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.081980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.082043 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.083044 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.083047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.083066 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:10.083160 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.083167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.084534 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.084537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.085493 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.085503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.086734 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.086738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.086825 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.086848 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.086899 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.086976 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 22:43:10.087214 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.082222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.26.189.193:33931)
0807 22:43:10.082533 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.082547 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.082572 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:10.082649 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.082655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.082879 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.083056 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:10.083260 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.084419 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.085611 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.086586 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.086759 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.086904 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.086966 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.087059 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":124,"threads_started":1}
I20260807 22:43:10.088151 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.082428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33201 (local address 127.0.0.1:54812)
0807 22:43:10.082673 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.082685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.082769 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.083863 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.083866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.083883 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:10.083976 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.083982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.085131 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.085134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.086546 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.086558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.087726 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.087729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.087823 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.087848 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.087928 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.087998 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":18,"thread_start_us":110,"threads_started":1}
I20260807 22:43:10.088183 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.082930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.26.189.195:33201)
0807 22:43:10.083281 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.083292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.083314 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:43:10.083381 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.083386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.083569 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.083726 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:10.084098 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.085009 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.086679 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.087608 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.087711 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.087795 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.087951 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.088057 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":89,"threads_started":1}
I20260807 22:43:10.089658 6683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.082050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:42137 (local address 127.0.0.1:57562)
0807 22:43:10.082328 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.082341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.082420 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.084642 (+ 2222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.084645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.084663 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:10.084747 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.084752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.085933 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.085937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.088113 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.088124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.089302 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.089306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.089393 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.089416 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.089463 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.089525 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":23,"thread_start_us":105,"threads_started":1}
I20260807 22:43:10.090361 6694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.082632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.26.189.194:42137)
0807 22:43:10.084242 (+ 1610us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.084255 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.084274 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:43:10.084338 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.084342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.084515 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.084661 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:10.084832 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.085810 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.088396 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.089165 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.089942 (+ 777us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.090066 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.090115 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.090200 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1509,"thread_start_us":133,"threads_started":1}
I20260807 22:43:10.090126 6145 tablet_service.cc:1467] Processing CreateTablet for tablet 78d75727057e45a0b46094fd7bc8ad8d (DEFAULT_TABLE table=test-table [id=8ab2842491984bbca394db02a66eb4a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:10.091208 6145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d75727057e45a0b46094fd7bc8ad8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:10.091269 6548 tablet_service.cc:1467] Processing CreateTablet for tablet 78d75727057e45a0b46094fd7bc8ad8d (DEFAULT_TABLE table=test-table [id=8ab2842491984bbca394db02a66eb4a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:10.092131 6548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d75727057e45a0b46094fd7bc8ad8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:10.092981 6348 tablet_service.cc:1467] Processing CreateTablet for tablet 78d75727057e45a0b46094fd7bc8ad8d (DEFAULT_TABLE table=test-table [id=8ab2842491984bbca394db02a66eb4a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:10.093978 6348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d75727057e45a0b46094fd7bc8ad8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:10.099007 6697 tablet_bootstrap.cc:492] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Bootstrap starting.
I20260807 22:43:10.099682 6698 tablet_bootstrap.cc:492] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Bootstrap starting.
I20260807 22:43:10.101253 6699 tablet_bootstrap.cc:492] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Bootstrap starting.
I20260807 22:43:10.101758 6698 tablet_bootstrap.cc:654] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:10.102090 6697 tablet_bootstrap.cc:654] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:10.102754 6698 log.cc:824] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:10.103341 6697 log.cc:824] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:10.103443 6699 tablet_bootstrap.cc:654] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:10.104246 6699 log.cc:824] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:10.104812 6698 tablet_bootstrap.cc:492] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: No bootstrap required, opened a new log
I20260807 22:43:10.105008 6698 ts_tablet_manager.cc:1397] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:10.105370 6697 tablet_bootstrap.cc:492] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: No bootstrap required, opened a new log
I20260807 22:43:10.105547 6697 ts_tablet_manager.cc:1397] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 22:43:10.105968 6699 tablet_bootstrap.cc:492] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: No bootstrap required, opened a new log
I20260807 22:43:10.106142 6699 ts_tablet_manager.cc:1397] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 22:43:10.109750 6698 raft_consensus.cc:348] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.110021 6698 raft_consensus.cc:374] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:10.109876 6697 raft_consensus.cc:348] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.110083 6697 raft_consensus.cc:374] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:10.110085 6698 raft_consensus.cc:729] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e6c93b0d4b4bb6b4964f4b2bd0b433, State: Initialized, Role: FOLLOWER
I20260807 22:43:10.110134 6697 raft_consensus.cc:729] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b02e13866fe4e898143821c82626f11, State: Initialized, Role: FOLLOWER
I20260807 22:43:10.110038 6699 raft_consensus.cc:348] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.110306 6699 raft_consensus.cc:374] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:10.110363 6699 raft_consensus.cc:729] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bc7776e4884366969cd99d4ef7648f, State: Initialized, Role: FOLLOWER
I20260807 22:43:10.110589 6698 consensus_queue.cc:260] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.110600 6697 consensus_queue.cc:260] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.110755 6699 consensus_queue.cc:260] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.111716 6699 ts_tablet_manager.cc:1428] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:43:10.112195 6241 heartbeater.cc:503] Master 127.26.189.254:39689 was elected leader, sending a full tablet report...
I20260807 22:43:10.112214 6450 heartbeater.cc:503] Master 127.26.189.254:39689 was elected leader, sending a full tablet report...
I20260807 22:43:10.112334 6643 heartbeater.cc:503] Master 127.26.189.254:39689 was elected leader, sending a full tablet report...
I20260807 22:43:10.113236 6698 ts_tablet_manager.cc:1428] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 22:43:10.117182 6697 ts_tablet_manager.cc:1428] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260807 22:43:10.119046 6706 raft_consensus.cc:482] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:10.119222 6706 raft_consensus.cc:504] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.120260 6706 leader_election.cc:288] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201), 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931)
I20260807 22:43:10.124092 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.120487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33201 (local address 127.26.189.194:33137)
0807 22:43:10.120586 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.120602 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.120672 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.120917 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.120919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.120928 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:10.121024 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.121030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.122136 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.122138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.123061 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.123072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.123699 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.123703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.123803 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.123825 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.123889 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.123957 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":1}
I20260807 22:43:10.124090 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.120556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:33137 (local address 127.26.189.195:33201)
0807 22:43:10.120756 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.120760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.120773 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:10.120817 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.120819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.120856 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.120914 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:10.121303 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.122034 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.123181 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.123595 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.123699 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.123752 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.123911 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.123982 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260807 22:43:10.124788 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.120488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:33931 (local address 127.26.189.194:47943)
0807 22:43:10.121095 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:10.121108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:10.121178 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:10.121378 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:10.121381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:10.121398 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:10.121491 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.121497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.122419 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.122423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.123488 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:10.123498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.124406 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.124410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:10.124514 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.124536 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:43:10.124611 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:43:10.124673 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":482}
I20260807 22:43:10.124810 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.120643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:47943 (local address 127.26.189.193:33931)
0807 22:43:10.120755 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.120760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.121169 (+ 409us) server_negotiation.cc:407] Connection header received
0807 22:43:10.121244 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.121246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.121290 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.121363 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:10.121621 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.122273 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.123638 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.124208 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.124311 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.124352 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.124629 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.124706 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:43:10.124790 6578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d75727057e45a0b46094fd7bc8ad8d" candidate_uuid: "67bc7776e4884366969cd99d4ef7648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b02e13866fe4e898143821c82626f11" is_pre_election: true
I20260807 22:43:10.125070 6578 raft_consensus.cc:2454] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bc7776e4884366969cd99d4ef7648f in term 0.
I20260807 22:43:10.125501 6273 leader_election.cc:302] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67bc7776e4884366969cd99d4ef7648f, 8b02e13866fe4e898143821c82626f11; no voters:
I20260807 22:43:10.125696 6171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d75727057e45a0b46094fd7bc8ad8d" candidate_uuid: "67bc7776e4884366969cd99d4ef7648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" is_pre_election: true
I20260807 22:43:10.125769 6706 raft_consensus.cc:2781] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:10.125836 6706 raft_consensus.cc:482] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:10.125872 6706 raft_consensus.cc:3037] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:10.125943 6171 raft_consensus.cc:2454] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bc7776e4884366969cd99d4ef7648f in term 0.
W20260807 22:43:10.126039 6243 tablet.cc:2369] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:10.127125 6706 raft_consensus.cc:504] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.127674 6706 leader_election.cc:288] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [CANDIDATE]: Term 1 election: Requested vote from peers 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201), 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931)
I20260807 22:43:10.127993 6171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d75727057e45a0b46094fd7bc8ad8d" candidate_uuid: "67bc7776e4884366969cd99d4ef7648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433"
I20260807 22:43:10.128104 6171 raft_consensus.cc:3037] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:10.129273 6171 raft_consensus.cc:2454] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bc7776e4884366969cd99d4ef7648f in term 1.
I20260807 22:43:10.129501 6578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d75727057e45a0b46094fd7bc8ad8d" candidate_uuid: "67bc7776e4884366969cd99d4ef7648f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b02e13866fe4e898143821c82626f11"
I20260807 22:43:10.129637 6578 raft_consensus.cc:3037] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:10.129706 6271 leader_election.cc:302] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e6c93b0d4b4bb6b4964f4b2bd0b433, 67bc7776e4884366969cd99d4ef7648f; no voters:
I20260807 22:43:10.129943 6706 raft_consensus.cc:2781] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:10.130205 6706 raft_consensus.cc:686] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 1 LEADER]: Becoming Leader. State: Replica: 67bc7776e4884366969cd99d4ef7648f, State: Running, Role: LEADER
I20260807 22:43:10.130540 6706 consensus_queue.cc:237] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } }
I20260807 22:43:10.130837 6578 raft_consensus.cc:2454] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bc7776e4884366969cd99d4ef7648f in term 1.
I20260807 22:43:10.133858 4713 catalog_manager.cc:5495] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f reported cstate change: term changed from 0 to 1, leader changed from <none> to 67bc7776e4884366969cd99d4ef7648f (127.26.189.194). New cstate: current_term: 1 leader_uuid: "67bc7776e4884366969cd99d4ef7648f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } health_report { overall_health: HEALTHY } } }
I20260807 22:43:10.147330 6694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.144229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.26.189.194:42137)
0807 22:43:10.144335 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.144339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.144351 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:10.144464 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.144466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.144514 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.144594 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:10.144779 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.145261 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.145995 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.146198 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.146261 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.146614 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:10.146988 (+ 374us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:10.147075 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.147136 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.147181 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:43:10.152781 6578 raft_consensus.cc:1261] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Refusing update from remote peer 67bc7776e4884366969cd99d4ef7648f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:10.152781 6171 raft_consensus.cc:1261] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Refusing update from remote peer 67bc7776e4884366969cd99d4ef7648f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:10.153579 6706 consensus_queue.cc:1035] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [LEADER]: Connected to new peer: Peer: permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:10.153785 6707 consensus_queue.cc:1035] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:10.162708 6714 mvcc.cc:205] Tried to move back new op lower bound from 7316040049258860544 to 7316040049177591808. Current Snapshot: MvccSnapshot[applied={T|T < 7316040049258860544}]
I20260807 22:43:10.164091 6715 mvcc.cc:205] Tried to move back new op lower bound from 7316040049258860544 to 7316040049177591808. Current Snapshot: MvccSnapshot[applied={T|T < 7316040049258860544}]
I20260807 22:43:10.172648 4712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:10.172791 4712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:10.177397 4712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9146e72314e44334b85ed4be5c622d1d: Sending DeleteTablet for 3 replicas of tablet 78d75727057e45a0b46094fd7bc8ad8d
I20260807 22:43:10.178061 6548 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d75727057e45a0b46094fd7bc8ad8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:10 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:54812
I20260807 22:43:10.178056 6145 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d75727057e45a0b46094fd7bc8ad8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:10 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:36894
I20260807 22:43:10.178177 6348 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d75727057e45a0b46094fd7bc8ad8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:10 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:57562
I20260807 22:43:10.178516 6725 tablet_replica.cc:330] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: stopping tablet replica
I20260807 22:43:10.178776 6725 raft_consensus.cc:2229] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:10.179036 6725 raft_consensus.cc:2258] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:10.179409 6729 tablet_replica.cc:330] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: stopping tablet replica
I20260807 22:43:10.179608 6729 raft_consensus.cc:2229] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:10.179898 6729 raft_consensus.cc:2258] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:10.181135 6725 ts_tablet_manager.cc:1905] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:10.183604 6725 ts_tablet_manager.cc:1918] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:10.183692 6725 log.cc:1196] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/78d75727057e45a0b46094fd7bc8ad8d
I20260807 22:43:10.184006 6725 ts_tablet_manager.cc:1939] T 78d75727057e45a0b46094fd7bc8ad8d P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Deleting consensus metadata
I20260807 22:43:10.185248 4688 catalog_manager.cc:4841] TS 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931): tablet 78d75727057e45a0b46094fd7bc8ad8d (table test-table [id=8ab2842491984bbca394db02a66eb4a6]) successfully deleted
I20260807 22:43:10.186328 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 4663
I20260807 22:43:10.196724 6729 ts_tablet_manager.cc:1905] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:10.198107 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.180328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.26.189.253:43053)
0807 22:43:10.180432 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.180437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.180449 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:10.180486 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.180489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.180544 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.180626 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.181480 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.182333 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.183407 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.183771 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.183866 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.185570 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.185574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.185576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.185602 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.196193 (+ 10591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.196693 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.196698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.196699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.196789 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.197155 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.197160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.197162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.197400 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.197547 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.197796 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.197930 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:43:10.199473 6729 ts_tablet_manager.cc:1918] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:10.199580 6729 log.cc:1196] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/78d75727057e45a0b46094fd7bc8ad8d
I20260807 22:43:10.199903 6729 ts_tablet_manager.cc:1939] T 78d75727057e45a0b46094fd7bc8ad8d P 67bc7776e4884366969cd99d4ef7648f: Deleting consensus metadata
I20260807 22:43:10.201256 6726 tablet_replica.cc:330] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: stopping tablet replica
I20260807 22:43:10.201594 6726 raft_consensus.cc:2229] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:10.201933 6726 raft_consensus.cc:2258] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 22:43:10.205634 6275 connection.cc:535] server connection from 127.0.0.1:57562 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:43:10.206627 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.180774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.26.189.252:43697)
0807 22:43:10.180878 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.180882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.198319 (+ 17437us) server_negotiation.cc:407] Connection header received
0807 22:43:10.198563 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.198567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.198633 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.198689 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.199420 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.200099 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.201107 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.201389 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.201476 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.203152 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.203156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.203157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.203182 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.204690 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.205109 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.205113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.205115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.205198 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.205546 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.205550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.205551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.205789 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.205904 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.206320 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.206452 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 22:43:10.208436 6726 ts_tablet_manager.cc:1905] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:10.211220 6726 ts_tablet_manager.cc:1918] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:10.211323 6726 log.cc:1196] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/78d75727057e45a0b46094fd7bc8ad8d
I20260807 22:43:10.211663 6726 ts_tablet_manager.cc:1939] T 78d75727057e45a0b46094fd7bc8ad8d P 8b02e13866fe4e898143821c82626f11: Deleting consensus metadata
W20260807 22:43:10.212786 6486 connection.cc:441] server connection from 127.0.0.1:54812 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54812 (request call id 1) could send its response
I20260807 22:43:10.223021 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.214909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.26.189.252:43697)
0807 22:43:10.215021 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.215026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.215417 (+ 391us) server_negotiation.cc:407] Connection header received
0807 22:43:10.215676 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.215679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.215726 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.215801 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.216580 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.217093 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.217874 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.218080 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.218171 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.219546 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.219550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.219552 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.219579 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.221101 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.221572 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.221576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.221578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.221670 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.221993 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.221996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.221998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.222220 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.222411 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.222753 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.222860 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:43:10.224217 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:10.215700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.26.189.253:43053)
0807 22:43:10.215814 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:10.215818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:10.216041 (+ 223us) server_negotiation.cc:407] Connection header received
0807 22:43:10.216136 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:10.216140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:10.216189 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:10.216272 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:10.217098 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.217875 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.219083 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:10.219429 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:10.219522 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:10.220927 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:10.220932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:10.220934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:10.220959 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:10.222494 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.222993 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.222996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.222998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.223089 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:10.223362 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:10.223365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:10.223367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:10.223574 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:10.223688 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:10.223970 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 22:43:10.224074 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:43:11.166924 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.166133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:50535)
0807 22:43:11.166455 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.166763 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":96,"threads_started":1}
W20260807 22:43:11.167317 6450 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:39689 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
I20260807 22:43:11.167714 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.166953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:52689)
0807 22:43:11.167533 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.167622 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":75,"threads_started":1}
I20260807 22:43:11.167976 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.167332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:49415)
0807 22:43:11.167654 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.167742 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":54,"threads_started":1}
W20260807 22:43:11.168373 6643 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:39689 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
W20260807 22:43:11.168509 6241 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:39689 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
I20260807 22:43:11.676733 6745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9146e72314e44334b85ed4be5c622d1d)
I20260807 22:43:11.676967 6745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } }
I20260807 22:43:11.677995 6746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9146e72314e44334b85ed4be5c622d1d)
I20260807 22:43:11.678115 6745 leader_election.cc:288] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689), 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252:43697)
I20260807 22:43:11.678371 6746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } }
I20260807 22:43:11.678784 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.678241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39750)
0807 22:43:11.678601 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.678683 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":64,"threads_started":1}
W20260807 22:43:11.678963 5655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
I20260807 22:43:11.679312 6746 leader_election.cc:288] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689), 600910733a9044d4b25b30003f4a2f86 (127.26.189.253:43053)
I20260807 22:43:11.679916 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.679663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39768)
0807 22:43:11.679798 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.679849 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 22:43:11.680132 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.679410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39756)
0807 22:43:11.679979 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.680048 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":67,"threads_started":1}
W20260807 22:43:11.680150 5655 leader_election.cc:334] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689): Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
W20260807 22:43:11.680271 5956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
I20260807 22:43:11.682410 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.682171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39778)
0807 22:43:11.682274 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.682346 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:43:11.682600 5956 leader_election.cc:334] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689): Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
I20260807 22:43:11.683022 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.678241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:43697 (local address 127.0.0.1:34838)
0807 22:43:11.678670 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.678685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.678776 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.679318 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.679321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.679342 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.679452 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.679459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.680455 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.680458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.681724 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.681732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.682571 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.682574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.682670 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.682694 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.682765 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.682905 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":107,"thread_start_us":55,"threads_started":1}
I20260807 22:43:11.683035 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.678467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.26.189.252:43697)
0807 22:43:11.678714 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.678718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.678735 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:11.678969 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.678972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.679013 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.679082 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.679592 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.680346 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.681830 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.682429 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.682537 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.682590 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.682835 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.682905 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":184,"thread_start_us":90,"threads_started":1}
I20260807 22:43:11.683527 5986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "600910733a9044d4b25b30003f4a2f86" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" is_pre_election: true
I20260807 22:43:11.683670 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.679759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55906 (local address 127.26.189.253:43053)
0807 22:43:11.680016 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.680024 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.680297 (+ 273us) server_negotiation.cc:407] Connection header received
0807 22:43:11.680336 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.680339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.680380 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.680454 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.680724 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.681470 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.682697 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.683214 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.683329 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.683374 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.683522 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.683575 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
I20260807 22:43:11.683705 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.679734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:43053 (local address 127.0.0.1:55906)
0807 22:43:11.680134 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.680147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.680230 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.680477 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.680480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.680501 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.680617 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.680625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.681557 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.681559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.682554 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.682565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.683320 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.683324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.683410 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.683435 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.683511 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.683579 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":23,"thread_start_us":60,"threads_started":1}
I20260807 22:43:11.683712 5986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600910733a9044d4b25b30003f4a2f86 in term 1.
I20260807 22:43:11.684118 5652 leader_election.cc:302] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e7e3a5bf79b4799b2d2836f1df5bc52, 600910733a9044d4b25b30003f4a2f86; no voters: 9146e72314e44334b85ed4be5c622d1d
I20260807 22:43:11.684271 5704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "600910733a9044d4b25b30003f4a2f86" is_pre_election: true
I20260807 22:43:11.684324 6745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 22:43:11.684394 6745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9146e72314e44334b85ed4be5c622d1d)
I20260807 22:43:11.684445 6745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:43:11.685535 6745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } }
I20260807 22:43:11.685748 5704 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1e7e3a5bf79b4799b2d2836f1df5bc52 in current term 2: Already voted for candidate 600910733a9044d4b25b30003f4a2f86 in this term.
I20260807 22:43:11.685863 6745 leader_election.cc:288] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [CANDIDATE]: Term 2 election: Requested vote from peers 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689), 1e7e3a5bf79b4799b2d2836f1df5bc52 (127.26.189.252:43697)
I20260807 22:43:11.686182 5986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "600910733a9044d4b25b30003f4a2f86" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52"
I20260807 22:43:11.686278 5986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:43:11.686321 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.686063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39794)
0807 22:43:11.686185 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.686235 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 22:43:11.686594 5956 leader_election.cc:302] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e7e3a5bf79b4799b2d2836f1df5bc52; no voters: 600910733a9044d4b25b30003f4a2f86, 9146e72314e44334b85ed4be5c622d1d
I20260807 22:43:11.686976 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.686762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39796)
0807 22:43:11.686881 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.686922 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 22:43:11.687088 5655 leader_election.cc:334] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689): Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
I20260807 22:43:11.687435 5986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600910733a9044d4b25b30003f4a2f86 in term 2.
I20260807 22:43:11.687539 6746 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:43:11.687688 5652 leader_election.cc:302] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e7e3a5bf79b4799b2d2836f1df5bc52, 600910733a9044d4b25b30003f4a2f86; no voters: 9146e72314e44334b85ed4be5c622d1d
I20260807 22:43:11.687844 6745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:43:11.688037 6745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [term 2 LEADER]: Becoming Leader. State: Replica: 600910733a9044d4b25b30003f4a2f86, State: Running, Role: LEADER
I20260807 22:43:11.688267 6745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } }
I20260807 22:43:11.689365 6765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 600910733a9044d4b25b30003f4a2f86. Latest consensus state: current_term: 2 leader_uuid: "600910733a9044d4b25b30003f4a2f86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:11.689459 6765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:11.689703 6767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:11.691787 6767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:11.692173 6767 catalog_manager.cc:1259] Loaded cluster ID: b6ef1092b6ac4220b882fa7b5465e75d
I20260807 22:43:11.692221 6767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:11.692759 6767 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:11.693132 6767 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Loaded TSK: 0
I20260807 22:43:11.693748 6767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:11.734376 5685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:11.737416 5685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:11.740257 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.740028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39824)
0807 22:43:11.740145 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.740192 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 22:43:11.740314 5986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [term 2 FOLLOWER]: Refusing update from remote peer 600910733a9044d4b25b30003f4a2f86: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 22:43:11.740809 6765 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 22:43:11.740885 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.740683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.0.0.1:39828)
0807 22:43:11.740786 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.740831 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 22:43:11.741117 5655 consensus_peers.cc:487] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 -> Peer 9146e72314e44334b85ed4be5c622d1d (127.26.189.254:39689): Couldn't send request to peer 9146e72314e44334b85ed4be5c622d1d. Status: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:43:11.742259 6746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 600910733a9044d4b25b30003f4a2f86. Latest consensus state: current_term: 2 leader_uuid: "600910733a9044d4b25b30003f4a2f86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:11.742416 6746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:11.743290 6768 mvcc.cc:205] Tried to move back new op lower bound from 7316040055763734528 to 7316040055557402624. Current Snapshot: MvccSnapshot[applied={T|T < 7316040055763734528}]
I20260807 22:43:11.743424 6745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "600910733a9044d4b25b30003f4a2f86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:11.743511 6745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:11.743813 6746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "600910733a9044d4b25b30003f4a2f86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9146e72314e44334b85ed4be5c622d1d" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39689 } } peers { permanent_uuid: "600910733a9044d4b25b30003f4a2f86" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 43053 } attrs { promote: false } } peers { permanent_uuid: "1e7e3a5bf79b4799b2d2836f1df5bc52" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 43697 } attrs { promote: false } } }
I20260807 22:43:11.743916 6746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7e3a5bf79b4799b2d2836f1df5bc52 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:43:11.744598 6769 mvcc.cc:205] Tried to move back new op lower bound from 7316040055763734528 to 7316040055557402624. Current Snapshot: MvccSnapshot[applied={T|T < 7316040055763734528 or (T in {7316040055763734528})}]
I20260807 22:43:11.753476 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.748962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:42137 (local address 127.0.0.1:57592)
0807 22:43:11.749312 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.749327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.749403 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.749839 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.749842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.749859 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.749962 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.749968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.751001 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.751004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.751847 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.751856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.753081 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.753085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.753173 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.753200 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.753272 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.753345 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260807 22:43:11.753489 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.749454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33201 (local address 127.0.0.1:54826)
0807 22:43:11.749707 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.749719 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.749800 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.750433 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.750436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.750456 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.750554 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.750560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.751602 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.751605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.752381 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.752390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.753176 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.753179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.753254 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.753273 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.753312 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.753365 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":17,"thread_start_us":99,"threads_started":1}
I20260807 22:43:11.753525 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.749071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.26.189.194:42137)
0807 22:43:11.749570 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.749575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.749591 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:43:11.749657 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.749660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.749712 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.750072 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.750196 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.750907 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.751985 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.752945 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.753086 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.753146 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.753295 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.753371 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":62,"threads_started":1}
I20260807 22:43:11.753844 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.749228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:33931 (local address 127.0.0.1:36908)
0807 22:43:11.749460 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.749472 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.749533 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.750394 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.750397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.750414 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.750507 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.750513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.751575 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.751578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.752683 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.752692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.753505 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.753509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.753619 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.753641 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.753691 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.753757 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":36}
I20260807 22:43:11.753921 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.749891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.26.189.195:33201)
0807 22:43:11.750156 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.750161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.750172 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:43:11.750216 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.750219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.750261 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.750399 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.750667 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.751468 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.752493 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.753045 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.753648 (+ 603us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.753698 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.753735 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.753798 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
I20260807 22:43:11.754139 6348 tablet_service.cc:1467] Processing CreateTablet for tablet 71bffe13c6f8464ea6d84c7e07499f9e (DEFAULT_TABLE table=test-table [id=edf2bb379acd4957beb21d6865ec3af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:11.754411 6548 tablet_service.cc:1467] Processing CreateTablet for tablet 71bffe13c6f8464ea6d84c7e07499f9e (DEFAULT_TABLE table=test-table [id=edf2bb379acd4957beb21d6865ec3af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:11.754465 6348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bffe13c6f8464ea6d84c7e07499f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:11.754693 6548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bffe13c6f8464ea6d84c7e07499f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:11.754724 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.749686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.26.189.193:33931)
0807 22:43:11.750117 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.750121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.750135 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:43:11.750186 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.750188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.750232 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.750604 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.750704 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.751459 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.752810 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.753376 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.754406 (+ 1030us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.754456 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.754490 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.754591 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":67,"threads_started":1}
I20260807 22:43:11.756809 6782 tablet_bootstrap.cc:492] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Bootstrap starting.
I20260807 22:43:11.756873 6783 tablet_bootstrap.cc:492] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Bootstrap starting.
I20260807 22:43:11.757336 6145 tablet_service.cc:1467] Processing CreateTablet for tablet 71bffe13c6f8464ea6d84c7e07499f9e (DEFAULT_TABLE table=test-table [id=edf2bb379acd4957beb21d6865ec3af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:11.757602 6782 tablet_bootstrap.cc:654] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:11.757606 6145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bffe13c6f8464ea6d84c7e07499f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:11.757728 6783 tablet_bootstrap.cc:654] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:11.758703 6782 tablet_bootstrap.cc:492] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: No bootstrap required, opened a new log
I20260807 22:43:11.758788 6782 ts_tablet_manager.cc:1397] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:11.759133 6782 raft_consensus.cc:348] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.759225 6782 raft_consensus.cc:374] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:11.759254 6782 raft_consensus.cc:729] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bc7776e4884366969cd99d4ef7648f, State: Initialized, Role: FOLLOWER
I20260807 22:43:11.759357 6782 consensus_queue.cc:260] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.759703 6782 ts_tablet_manager.cc:1428] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:11.760593 6783 tablet_bootstrap.cc:492] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: No bootstrap required, opened a new log
I20260807 22:43:11.760689 6783 ts_tablet_manager.cc:1397] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 22:43:11.760694 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.760245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:54575)
0807 22:43:11.760496 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.760554 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260807 22:43:11.761066 6783 raft_consensus.cc:348] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.761188 6783 raft_consensus.cc:374] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:11.761227 6783 raft_consensus.cc:729] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b02e13866fe4e898143821c82626f11, State: Initialized, Role: FOLLOWER
I20260807 22:43:11.761355 6783 consensus_queue.cc:260] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.761736 6783 ts_tablet_manager.cc:1428] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:11.762534 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.762121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:56791)
0807 22:43:11.762390 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.762447 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260807 22:43:11.762904 6451 heartbeater.cc:503] Master 127.26.189.253:43053 was elected leader, sending a full tablet report...
I20260807 22:43:11.762996 6788 tablet_bootstrap.cc:492] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Bootstrap starting.
I20260807 22:43:11.763285 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.763029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:53217)
0807 22:43:11.763181 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.763225 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 22:43:11.763634 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.763411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:37747)
0807 22:43:11.763518 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.763569 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 22:43:11.763660 6645 heartbeater.cc:503] Master 127.26.189.253:43053 was elected leader, sending a full tablet report...
I20260807 22:43:11.764130 6788 tablet_bootstrap.cc:654] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:11.765188 6788 tablet_bootstrap.cc:492] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433: No bootstrap required, opened a new log
I20260807 22:43:11.765295 6788 ts_tablet_manager.cc:1397] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:11.765650 6788 raft_consensus.cc:348] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.765762 6788 raft_consensus.cc:374] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:11.765802 6788 raft_consensus.cc:729] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e6c93b0d4b4bb6b4964f4b2bd0b433, State: Initialized, Role: FOLLOWER
I20260807 22:43:11.765903 6788 consensus_queue.cc:260] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.766203 6788 ts_tablet_manager.cc:1428] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:11.767019 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.766649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:38037)
0807 22:43:11.766867 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.766922 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260807 22:43:11.767860 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.767654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:60631)
0807 22:43:11.767751 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.767802 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:43:11.767887 6242 heartbeater.cc:503] Master 127.26.189.253:43053 was elected leader, sending a full tablet report...
W20260807 22:43:11.784402 6646 tablet.cc:2369] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:11.786345 6796 raft_consensus.cc:482] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:11.786446 6796 raft_consensus.cc:504] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.787529 6796 leader_election.cc:288] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137), 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201)
I20260807 22:43:11.792464 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.787740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:50357 (local address 127.26.189.194:42137)
0807 22:43:11.787852 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.787856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.788097 (+ 241us) server_negotiation.cc:407] Connection header received
0807 22:43:11.788218 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.788221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.788267 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.788342 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.788637 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.789408 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.791230 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.791827 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.791941 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.791992 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.792279 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.792347 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:43:11.792464 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.787882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:35259 (local address 127.26.189.195:33201)
0807 22:43:11.787995 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.788000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.788099 (+ 99us) server_negotiation.cc:407] Connection header received
0807 22:43:11.788218 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.788221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.788268 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.788342 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:11.788637 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.789408 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.791230 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.791827 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.791941 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.791992 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.792281 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.792347 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:43:11.792531 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.787744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33201 (local address 127.26.189.193:35259)
0807 22:43:11.788020 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.788031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.788133 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.788419 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.788421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.788434 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.788536 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.788543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.789502 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.789505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.791113 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.791125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.791951 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.791956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.792143 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.792181 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.792253 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.792364 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":233,"spinlock_wait_cycles":12032,"thread_start_us":103,"threads_started":1}
I20260807 22:43:11.792694 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.787677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:42137 (local address 127.26.189.193:50357)
0807 22:43:11.787892 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.787905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:11.788133 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:11.788364 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:11.788366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:11.788386 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:11.788536 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.788543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.789502 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.789505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.791113 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:11.791125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.791962 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.791965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:11.792139 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.792185 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:43:11.792256 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:43:11.792364 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":289,"spinlock_wait_cycles":3840}
I20260807 22:43:11.792929 6578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bffe13c6f8464ea6d84c7e07499f9e" candidate_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b02e13866fe4e898143821c82626f11" is_pre_election: true
I20260807 22:43:11.793053 6578 raft_consensus.cc:2454] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05e6c93b0d4b4bb6b4964f4b2bd0b433 in term 0.
I20260807 22:43:11.793231 6383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bffe13c6f8464ea6d84c7e07499f9e" candidate_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67bc7776e4884366969cd99d4ef7648f" is_pre_election: true
I20260807 22:43:11.793411 6071 leader_election.cc:302] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05e6c93b0d4b4bb6b4964f4b2bd0b433, 8b02e13866fe4e898143821c82626f11; no voters:
I20260807 22:43:11.793431 6383 raft_consensus.cc:2454] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05e6c93b0d4b4bb6b4964f4b2bd0b433 in term 0.
I20260807 22:43:11.793623 6796 raft_consensus.cc:2781] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:11.793680 6796 raft_consensus.cc:482] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:11.793725 6796 raft_consensus.cc:3037] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:11.794724 6796 raft_consensus.cc:504] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.795015 6796 leader_election.cc:288] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [CANDIDATE]: Term 1 election: Requested vote from peers 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137), 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201)
I20260807 22:43:11.795310 6383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bffe13c6f8464ea6d84c7e07499f9e" candidate_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67bc7776e4884366969cd99d4ef7648f"
I20260807 22:43:11.795331 6578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bffe13c6f8464ea6d84c7e07499f9e" candidate_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b02e13866fe4e898143821c82626f11"
I20260807 22:43:11.795428 6578 raft_consensus.cc:3037] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:11.795428 6383 raft_consensus.cc:3037] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:11.796461 6578 raft_consensus.cc:2454] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05e6c93b0d4b4bb6b4964f4b2bd0b433 in term 1.
I20260807 22:43:11.796461 6383 raft_consensus.cc:2454] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05e6c93b0d4b4bb6b4964f4b2bd0b433 in term 1.
I20260807 22:43:11.796742 6075 leader_election.cc:302] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05e6c93b0d4b4bb6b4964f4b2bd0b433, 67bc7776e4884366969cd99d4ef7648f, 8b02e13866fe4e898143821c82626f11; no voters:
I20260807 22:43:11.796904 6796 raft_consensus.cc:2781] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:11.797137 6796 raft_consensus.cc:686] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 LEADER]: Becoming Leader. State: Replica: 05e6c93b0d4b4bb6b4964f4b2bd0b433, State: Running, Role: LEADER
I20260807 22:43:11.797374 6796 consensus_queue.cc:237] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } }
I20260807 22:43:11.798033 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.797718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:47397)
0807 22:43:11.797919 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.797969 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:43:11.799710 5682 catalog_manager.cc:5495] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05e6c93b0d4b4bb6b4964f4b2bd0b433" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 33931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:11.818121 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.814858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36918 (local address 127.26.189.193:33931)
0807 22:43:11.814967 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:11.814971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:11.814984 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:11.815132 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:11.815135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:11.815188 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:11.815274 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:11.815496 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.815992 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.816793 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:11.817002 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:11.817089 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:11.817342 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:11.817774 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:11.817875 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:11.817943 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 22:43:11.817996 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:43:11.820603 6578 raft_consensus.cc:1261] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Refusing update from remote peer 05e6c93b0d4b4bb6b4964f4b2bd0b433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:11.820814 6383 raft_consensus.cc:1261] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Refusing update from remote peer 05e6c93b0d4b4bb6b4964f4b2bd0b433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:11.821062 6800 consensus_queue.cc:1035] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b02e13866fe4e898143821c82626f11" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:11.821282 6796 consensus_queue.cc:1035] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67bc7776e4884366969cd99d4ef7648f" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:11.822741 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.822465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:33767)
0807 22:43:11.822583 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.822664 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 22:43:11.824285 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.823940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:55441)
0807 22:43:11.824162 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.824217 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260807 22:43:11.826362 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.826130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.193:48821)
0807 22:43:11.826213 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.826264 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 22:43:11.827193 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.826940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.195:36979)
0807 22:43:11.827079 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.827131 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 22:43:11.828883 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.828564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39689 (local address 127.26.189.194:60583)
0807 22:43:11.828778 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.828828 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 22:43:11.828707 6816 mvcc.cc:205] Tried to move back new op lower bound from 7316040056092942336 to 7316040056004939776. Current Snapshot: MvccSnapshot[applied={T|T < 7316040056092942336}]
I20260807 22:43:11.834082 5682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:11.834246 5682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:11.838771 5682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 600910733a9044d4b25b30003f4a2f86: Sending DeleteTablet for 3 replicas of tablet 71bffe13c6f8464ea6d84c7e07499f9e
I20260807 22:43:11.839319 6348 tablet_service.cc:1514] Processing DeleteTablet for tablet 71bffe13c6f8464ea6d84c7e07499f9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:11 UTC) from {username='oryx', principal='oryx/127.26.189.253@KRBTEST.COM'} at 127.0.0.1:57592
I20260807 22:43:11.839957 6145 tablet_service.cc:1514] Processing DeleteTablet for tablet 71bffe13c6f8464ea6d84c7e07499f9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:11 UTC) from {username='oryx', principal='oryx/127.26.189.253@KRBTEST.COM'} at 127.0.0.1:36908
I20260807 22:43:11.840159 6548 tablet_service.cc:1514] Processing DeleteTablet for tablet 71bffe13c6f8464ea6d84c7e07499f9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:11 UTC) from {username='oryx', principal='oryx/127.26.189.253@KRBTEST.COM'} at 127.0.0.1:54826
I20260807 22:43:11.840384 6827 tablet_replica.cc:330] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433: stopping tablet replica
I20260807 22:43:11.840529 6827 raft_consensus.cc:2229] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:11.840833 6827 raft_consensus.cc:2258] T 71bffe13c6f8464ea6d84c7e07499f9e P 05e6c93b0d4b4bb6b4964f4b2bd0b433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:11.840852 6823 tablet_replica.cc:330] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: stopping tablet replica
I20260807 22:43:11.840962 6823 raft_consensus.cc:2229] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:11.841075 6823 raft_consensus.cc:2258] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:11.841432 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 5922
I20260807 22:43:11.842185 6823 ts_tablet_manager.cc:1905] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:11.845052 6823 ts_tablet_manager.cc:1918] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:11.845135 6823 log.cc:1196] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/71bffe13c6f8464ea6d84c7e07499f9e
I20260807 22:43:11.845252 6828 tablet_replica.cc:330] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: stopping tablet replica
I20260807 22:43:11.845373 6828 raft_consensus.cc:2229] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:11.845448 6823 ts_tablet_manager.cc:1939] T 71bffe13c6f8464ea6d84c7e07499f9e P 67bc7776e4884366969cd99d4ef7648f: Deleting consensus metadata
I20260807 22:43:11.845499 6828 raft_consensus.cc:2258] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:11.846176 6828 ts_tablet_manager.cc:1905] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:11.846206 5656 catalog_manager.cc:4841] TS 67bc7776e4884366969cd99d4ef7648f (127.26.189.194:42137): tablet 71bffe13c6f8464ea6d84c7e07499f9e (table test-table [id=edf2bb379acd4957beb21d6865ec3af4]) successfully deleted
I20260807 22:43:11.848393 6828 ts_tablet_manager.cc:1918] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:11.848472 6828 log.cc:1196] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/71bffe13c6f8464ea6d84c7e07499f9e
I20260807 22:43:11.848721 6828 ts_tablet_manager.cc:1939] T 71bffe13c6f8464ea6d84c7e07499f9e P 8b02e13866fe4e898143821c82626f11: Deleting consensus metadata
I20260807 22:43:11.849318 5652 catalog_manager.cc:4841] TS 8b02e13866fe4e898143821c82626f11 (127.26.189.195:33201): tablet 71bffe13c6f8464ea6d84c7e07499f9e (table test-table [id=edf2bb379acd4957beb21d6865ec3af4]) successfully deleted
I20260807 22:43:11.851603 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:11.851373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:33931 (local address 127.0.0.1:36930)
0807 22:43:11.851474 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:11.851520 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:33931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 22:43:11.851635 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 6250
W20260807 22:43:11.851781 5650 catalog_manager.cc:4584] TS 05e6c93b0d4b4bb6b4964f4b2bd0b433 (127.26.189.193:33931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71bffe13c6f8464ea6d84c7e07499f9e: Network error: Client connection negotiation failed: client connection to 127.26.189.193:33931: connect: Connection refused (error 111)
I20260807 22:43:11.863776 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 6456
I20260807 22:43:11.873195 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 5482
I20260807 22:43:11.882519 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 5640
2026-08-07T22:43:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5296 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:11 dist-test-slave-4tdj krb5kdc[6831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:11 dist-test-slave-4tdj krb5kdc[6831](info): set up 2 sockets
Aug 07 22:43:11 dist-test-slave-4tdj krb5kdc[6831](info): commencing operation
krb5kdc: starting...
W20260807 22:43:13.024315 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab.
Entry for principal oryx/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab.
I20260807 22:43:13.145715 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:44769
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab
--principal=oryx/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:44769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:13.252077 6852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:13.252319 6852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:13.252372 6852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:13.256281 6852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:13.256330 6852 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:43:13.256353 6852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:13.256376 6852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:13.256402 6852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:13.256431 6852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:13.261426 6852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:44769
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.254.keytab
--principal=oryx/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:44769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:13.262377 6852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:13.263334 6852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:13.271392 6867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:13.271387 6862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.271469 6852 server_base.cc:1034] running on GCE node
W20260807 22:43:13.271389 6863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.272223 6852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:13.273249 6852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:13.274351 6852 hybrid_clock.cc:648] HybridClock initialized: now 1786142593274337 us; error 27 us; skew 500 ppm
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:13.276854 6852 init.cc:377] Logged in from keytab as oryx/127.26.189.254@KRBTEST.COM (short username oryx)
I20260807 22:43:13.277688 6852 webserver.cc:460] Webserver started at http://127.26.189.254:32771/ using document root <none> and password file <none>
I20260807 22:43:13.278192 6852 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:13.278258 6852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:13.278483 6852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:13.280433 6852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "21910e46b5ab5e69d3383cc63ededf15"
server_key_iv: "99db04c67ec5a23154b745aad09ae630"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.280970 6852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "21910e46b5ab5e69d3383cc63ededf15"
server_key_iv: "99db04c67ec5a23154b745aad09ae630"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.284495 6852 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:43:13.286799 6878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.287966 6852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:13.288074 6852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "21910e46b5ab5e69d3383cc63ededf15"
server_key_iv: "99db04c67ec5a23154b745aad09ae630"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.288173 6852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:13.300246 6852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:13.302903 6852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:13.303061 6852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:13.310704 6852 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:44769
I20260807 22:43:13.310742 6969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:44769 every 8 connection(s)
I20260807 22:43:13.311621 6852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:13.314327 6970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:13.319855 6970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Bootstrap starting.
I20260807 22:43:13.321179 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 6852
I20260807 22:43:13.321300 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:13.321565 27383 external_mini_cluster.cc:1428] Setting key 0bbb246c9f817443f91216ec14f4f53f
I20260807 22:43:13.322532 6970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:13.323410 6970 log.cc:824] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:13.325587 6970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: No bootstrap required, opened a new log
I20260807 22:43:13.329743 6970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44769 } }
I20260807 22:43:13.329967 6970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:13.330004 6970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5727fda8f7344b00a1ffe22d7d0a0ca3, State: Initialized, Role: FOLLOWER
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:13.330467 6970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44769 } }
I20260807 22:43:13.330587 6970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:13.330652 6970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:13.330746 6970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:13.331827 6970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44769 } }
I20260807 22:43:13.332211 6970 leader_election.cc:302] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5727fda8f7344b00a1ffe22d7d0a0ca3; no voters:
I20260807 22:43:13.332571 6970 leader_election.cc:288] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:13.332674 6978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:13.332911 6978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 5727fda8f7344b00a1ffe22d7d0a0ca3, State: Running, Role: LEADER
I20260807 22:43:13.333241 6978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44769 } }
I20260807 22:43:13.333915 6970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:13.334884 6981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5727fda8f7344b00a1ffe22d7d0a0ca3. Latest consensus state: current_term: 1 leader_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44769 } } }
I20260807 22:43:13.335005 6981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:13.335216 6980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5727fda8f7344b00a1ffe22d7d0a0ca3" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44769 } } }
I20260807 22:43:13.335314 6980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:13.335433 6996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:13.337167 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.322642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.26.189.254:44769)
0807 22:43:13.323122 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.323142 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.323188 (+ 46us) server_negotiation.cc:407] Connection header received
0807 22:43:13.323998 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.324020 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.324409 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.324777 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.325452 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.326957 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.327804 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.328525 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.328689 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.331153 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.331159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.331176 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.331205 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.333130 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.333669 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.333674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.333679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.333819 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.334160 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.334163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.334165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.334755 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.335696 (+ 941us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.335865 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.336183 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":325,"thread_start_us":152,"threads_started":1}
I20260807 22:43:13.338833 6996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:13.344605 6996 catalog_manager.cc:1347] Generated new cluster ID: 976672eb22bd4752b65a1fd81a358881
I20260807 22:43:13.344730 6996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:13.355520 6996 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:13.356417 6996 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:13.364035 6996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Generated new TSK 0
I20260807 22:43:13.364698 6996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab.
Entry for principal oryx/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab.
I20260807 22:43:13.394944 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44769
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:13.504793 7012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:13.505045 7012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:13.505098 7012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:13.509171 7012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:13.509238 7012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:13.509275 7012 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:13.509326 7012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:13.514725 7012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:13.515781 7012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:13.516799 7012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:13.526212 7025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:13.526227 7023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.526255 7012 server_base.cc:1034] running on GCE node
W20260807 22:43:13.526222 7027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.526796 7012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:13.527431 7012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:13.528585 7012 hybrid_clock.cc:648] HybridClock initialized: now 1786142593528572 us; error 31 us; skew 500 ppm
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:13.531234 7012 init.cc:377] Logged in from keytab as oryx/127.26.189.193@KRBTEST.COM (short username oryx)
I20260807 22:43:13.532084 7012 webserver.cc:460] Webserver started at http://127.26.189.193:43627/ using document root <none> and password file <none>
I20260807 22:43:13.532616 7012 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:13.532683 7012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:13.532878 7012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:13.534971 7012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "ec5943a2df9ec4adf0d0075691c687f7"
server_key_iv: "60c63687ed0ea659bc36a238a91843d8"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.535512 7012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "ec5943a2df9ec4adf0d0075691c687f7"
server_key_iv: "60c63687ed0ea659bc36a238a91843d8"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.539065 7012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:13.541270 7037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.542425 7012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:13.542522 7012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "ec5943a2df9ec4adf0d0075691c687f7"
server_key_iv: "60c63687ed0ea659bc36a238a91843d8"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.542622 7012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:13.553251 7012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:13.556160 7012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:13.556334 7012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:13.557694 7012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:13.557741 7012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.557781 7012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:13.557811 7012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:13.567656 7012 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:45533
I20260807 22:43:13.567690 7218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:45533 every 8 connection(s)
I20260807 22:43:13.568699 7012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:13.570940 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.558457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:60335 (local address 127.26.189.254:44769)
0807 22:43:13.558608 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.558613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.559403 (+ 790us) server_negotiation.cc:407] Connection header received
0807 22:43:13.560511 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.560515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.560574 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.560660 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.562059 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.562729 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.563915 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.564289 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.564416 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.567487 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.567491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.567494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.567525 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.568997 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.569578 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.569581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.569582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.569672 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.569947 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.569949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.569950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.570133 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.570208 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.570714 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.570821 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:43:13.571161 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 7012
I20260807 22:43:13.571295 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:13.571588 27383 external_mini_cluster.cc:1428] Setting key c6736988f5b4ee87dafa2d7cbbecaddd
I20260807 22:43:13.572945 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.558805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.26.189.193:60335)
0807 22:43:13.559247 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:13.559296 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:13.560274 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:13.560844 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:13.560853 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:13.561286 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:13.561908 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.561926 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.562844 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.562848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.563797 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.563808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.564473 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.564478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.564665 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.565736 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:13.565758 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:13.567275 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:13.569113 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.569119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.569134 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.569447 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.569740 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.569743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.569744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.569873 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.570219 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:13.570224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:13.570492 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 22:43:13.570655 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 22:43:13.570904 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260807 22:43:13.574110 7220 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44769
I20260807 22:43:13.574258 7220 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:13.574739 7220 heartbeater.cc:511] Master 127.26.189.254:44769 requested a full tablet report, sending...
I20260807 22:43:13.576310 6900 ts_manager.cc:194] Registered new tserver with Master: 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533)
I20260807 22:43:13.577728 6900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:60335
WARNING: no policy specified for oryx/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:43:13.583664 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.576687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.26.189.193:39187)
0807 22:43:13.576963 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:13.576978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:13.577075 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:13.577302 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:13.577305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:13.577510 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:13.577631 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.577638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.578689 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.578692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.579263 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.579272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.579819 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.579822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.579894 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.580331 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:13.580343 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:13.580657 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:13.582342 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.582345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.582347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.582525 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.582788 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.582790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.582791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.582855 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.583257 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:13.583259 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:13.583336 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 22:43:13.583415 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:43:13.583516 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260807 22:43:13.583685 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.576551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:39187 (local address 127.26.189.254:44769)
0807 22:43:13.576666 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.576671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.577056 (+ 385us) server_negotiation.cc:407] Connection header received
0807 22:43:13.577158 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.577161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.577204 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.577278 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.577756 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.578530 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.579375 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.579699 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.579805 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.580788 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.580792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.580796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.580820 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.582221 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.582619 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.582623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.582625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.582710 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.582922 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.582925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.582927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.583144 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.583272 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.583483 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.583577 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab.
Entry for principal oryx/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab.
I20260807 22:43:13.600929 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44769
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:13.708420 7238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:13.708657 7238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:13.708698 7238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:13.712679 7238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:13.712735 7238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:13.712771 7238 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:13.712811 7238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:13.717928 7238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:13.718928 7238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:13.719906 7238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:13.729172 7246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:13.729149 7243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.729231 7238 server_base.cc:1034] running on GCE node
W20260807 22:43:13.729143 7244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.729892 7238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:13.730540 7238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:13.731717 7238 hybrid_clock.cc:648] HybridClock initialized: now 1786142593731701 us; error 31 us; skew 500 ppm
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:13.734403 7238 init.cc:377] Logged in from keytab as oryx/127.26.189.194@KRBTEST.COM (short username oryx)
I20260807 22:43:13.735316 7238 webserver.cc:460] Webserver started at http://127.26.189.194:37863/ using document root <none> and password file <none>
I20260807 22:43:13.735867 7238 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:13.735947 7238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:13.736129 7238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:13.738138 7238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "fd306189caa2489484baaffe9380186b"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "46690643957289a3cc7fffa357443938"
server_key_iv: "c5def89a62949e725a59e550e0560fbc"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.738705 7238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "fd306189caa2489484baaffe9380186b"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "46690643957289a3cc7fffa357443938"
server_key_iv: "c5def89a62949e725a59e550e0560fbc"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.742151 7238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:13.744422 7258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.745612 7238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:13.745705 7238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "fd306189caa2489484baaffe9380186b"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "46690643957289a3cc7fffa357443938"
server_key_iv: "c5def89a62949e725a59e550e0560fbc"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.745782 7238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:13.758276 7238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:13.761255 7238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:13.761423 7238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:13.762970 7238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:13.763031 7238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.763089 7238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:13.763128 7238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:13.772816 7238 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:44751
I20260807 22:43:13.772866 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:44751 every 8 connection(s)
I20260807 22:43:13.773825 7238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:13.776561 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.763713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:38653 (local address 127.26.189.254:44769)
0807 22:43:13.763868 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.763873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.764795 (+ 922us) server_negotiation.cc:407] Connection header received
0807 22:43:13.765808 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.765812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.765874 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.765931 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.767620 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.768423 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.770111 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.770482 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.770602 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.773380 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.773384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.773387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.773418 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.774930 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.775355 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.775358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.775359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.775433 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.775711 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.775713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.775714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.775882 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.775954 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.776347 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.776442 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:43:13.777029 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 7238
I20260807 22:43:13.777148 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:13.777413 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.764165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.26.189.194:38653)
0807 22:43:13.764627 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:13.764676 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:13.765566 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:13.766374 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:13.766383 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:13.766824 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:13.767456 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.767474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.768564 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.768569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.769971 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.769983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.770639 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.770642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.770800 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.771383 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:13.771405 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:13.773211 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:13.775028 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.775032 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.775042 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.775282 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.775502 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.775505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.775506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.775640 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.775955 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:13.775960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:13.776169 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 22:43:13.776313 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:43:13.776539 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260807 22:43:13.777447 27383 external_mini_cluster.cc:1428] Setting key 6c432c69bf58a389e655d5897d6e1312
I20260807 22:43:13.778653 7455 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44769
I20260807 22:43:13.778802 7455 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:13.779332 7455 heartbeater.cc:511] Master 127.26.189.254:44769 requested a full tablet report, sending...
I20260807 22:43:13.780397 6901 ts_manager.cc:194] Registered new tserver with Master: fd306189caa2489484baaffe9380186b (127.26.189.194:44751)
I20260807 22:43:13.781093 6901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:38653
WARNING: no policy specified for oryx/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:43:13.788182 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.780673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.26.189.194:56711)
0807 22:43:13.781054 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:13.781066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:13.781141 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:13.781650 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:13.781653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:13.782502 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:13.782640 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.782646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.783368 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.783372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.783916 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.783924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.784340 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.784344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.784432 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.784860 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:13.784872 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:13.785175 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:13.786771 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.786774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.786776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.786962 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.787222 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.787225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.787227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.787306 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.787684 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:13.787687 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:13.787772 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 22:43:13.787868 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:43:13.787997 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":184,"threads_started":1}
I20260807 22:43:13.788213 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.781302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:56711 (local address 127.26.189.254:44769)
0807 22:43:13.781428 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.781432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.781444 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:13.781475 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.781478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.781520 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.781601 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.782756 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.783251 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.784021 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.784243 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.784325 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.785316 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.785320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.785321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.785342 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.786672 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.787058 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.787060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.787062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.787136 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.787383 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.787386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.787388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.787591 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.787678 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.787966 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.788084 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab.
Entry for principal oryx/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab.
I20260807 22:43:13.805605 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44769
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:13.918072 7469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:13.918326 7469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:13.918378 7469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:13.922322 7469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:13.922374 7469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:13.922415 7469 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:43:13.922466 7469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:13.927683 7469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.189.254:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:13.928617 7469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:13.929562 7469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:13.938776 7476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:13.938761 7474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.938939 7469 server_base.cc:1034] running on GCE node
W20260807 22:43:13.938761 7479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:13.939484 7469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:13.940093 7469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:13.941251 7469 hybrid_clock.cc:648] HybridClock initialized: now 1786142593941231 us; error 30 us; skew 500 ppm
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:13.943830 7469 init.cc:377] Logged in from keytab as oryx/127.26.189.195@KRBTEST.COM (short username oryx)
I20260807 22:43:13.944605 7469 webserver.cc:460] Webserver started at http://127.26.189.195:41213/ using document root <none> and password file <none>
I20260807 22:43:13.945086 7469 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:13.945144 7469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:13.945358 7469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:13.947454 7469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "cdc71316f9b9569121b8953ffba2531a"
server_key_iv: "31aa99925a0530539ea8c78d19f3927b"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.947962 7469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "cdc71316f9b9569121b8953ffba2531a"
server_key_iv: "31aa99925a0530539ea8c78d19f3927b"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.951615 7469 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:13.953897 7487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.955052 7469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:13.955150 7469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
format_stamp: "Formatted at 2026-08-07 22:43:13 on dist-test-slave-4tdj"
server_key: "cdc71316f9b9569121b8953ffba2531a"
server_key_iv: "31aa99925a0530539ea8c78d19f3927b"
server_key_version: "encryptionkey@0"
I20260807 22:43:13.955245 7469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:13.970311 7469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:13.972851 7469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:13.972997 7469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:13.974344 7469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:13.974396 7469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:13.974437 7469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:13.974465 7469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:43:13 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:13.984079 7469 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:35695
I20260807 22:43:13.984234 7649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:35695 every 8 connection(s)
I20260807 22:43:13.984925 7469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:13.987907 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.975127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:39391 (local address 127.26.189.254:44769)
0807 22:43:13.975262 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.975267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.976308 (+ 1041us) server_negotiation.cc:407] Connection header received
0807 22:43:13.977418 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.977423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.977481 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.977544 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.979185 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.979919 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.981577 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.981799 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.981885 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.984682 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.984686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.984689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.984716 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.986224 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.986716 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.986719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.986720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.986795 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.987047 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.987048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.987050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.987212 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.987280 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.987702 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.987790 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:43:13.988600 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.975670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.26.189.195:39391)
0807 22:43:13.976167 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:13.976206 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:13.977162 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:13.977964 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:13.977973 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:13.978402 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:13.979038 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.979058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.980044 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.980048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.981426 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.981438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.981935 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.981939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.982135 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.982763 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:13.982785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:13.984526 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:13.986345 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.986350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.986360 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.986635 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.986860 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.986862 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.986863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.986977 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.987278 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:13.987282 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:13.987515 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:43:13.987657 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 22:43:13.987875 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":145,"threads_started":1}
I20260807 22:43:13.989645 7650 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44769
I20260807 22:43:13.989779 7650 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:13.990337 7650 heartbeater.cc:511] Master 127.26.189.254:44769 requested a full tablet report, sending...
I20260807 22:43:13.991335 6900 ts_manager.cc:194] Registered new tserver with Master: 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:13.991878 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 7469
I20260807 22:43:13.991986 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:13.992039 6900 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:39391
I20260807 22:43:13.992303 27383 external_mini_cluster.cc:1428] Setting key e7ed393cd3937cbb0b92bf15d1887930
I20260807 22:43:13.994377 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:43:13.998787 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.991884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:35105 (local address 127.26.189.254:44769)
0807 22:43:13.991983 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:13.991986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:13.992484 (+ 498us) server_negotiation.cc:407] Connection header received
0807 22:43:13.992581 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:13.992584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:13.992627 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:13.992700 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:13.993168 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.993762 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.994652 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.994869 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.994951 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.995891 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:13.995895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:13.995897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:13.995921 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:13.997333 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.997741 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.997744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.997746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.997820 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:13.998076 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:13.998078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:13.998079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:13.998221 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:13.998311 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:13.998603 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 22:43:13.998687 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:43:13.998836 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:13.992086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.26.189.195:35105)
0807 22:43:13.992407 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:13.992421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:13.992507 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:13.992737 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:13.992740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:13.992937 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:13.993064 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.993072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.993881 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.993884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.994553 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:13.994562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:13.994972 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:13.994976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:13.995073 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:13.995476 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:13.995490 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:13.995768 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:13.997435 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.997438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.997439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.997651 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.997908 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:13.997911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:13.997912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:13.998002 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:13.998338 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:13.998341 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:13.998429 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 22:43:13.998541 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:43:13.998662 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":105,"threads_started":1}
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:43:14.018668 6973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:14.013213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.26.189.254:44769)
0807 22:43:14.013323 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.013326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.013337 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:43:14.013370 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.013372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.013405 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.013467 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:14.014079 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.014602 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.015386 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.015585 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.015661 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.018514 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53482: BlockingRecv error: recv got EOF from 127.0.0.1:53482 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:14.026625 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.019888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53490 (local address 127.26.189.254:44769)
0807 22:43:14.019983 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.019986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.020178 (+ 192us) server_negotiation.cc:407] Connection header received
0807 22:43:14.020256 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.020258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.020294 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.020352 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:14.020845 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.021302 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.021988 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.022178 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.022240 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.024240 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:14.024242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:14.024244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:14.024266 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:14.025356 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:14.025730 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:14.025732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:14.025733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:14.025799 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:14.025975 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:14.025977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:14.025978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:14.026121 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:14.026216 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.026391 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.026475 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:43:14.028985 6900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:14.031873 6900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:14.044881 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.039620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:35695 (local address 127.0.0.1:53294)
0807 22:43:14.039939 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.039989 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.040114 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.041055 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.041061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.041081 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.041240 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.041248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.042415 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.042418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.043325 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.043334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.044356 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.044360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.044472 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.044516 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.044624 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.044738 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":19,"thread_start_us":67,"threads_started":1}
I20260807 22:43:14.044888 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.039772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.26.189.195:35695)
0807 22:43:14.040416 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.040432 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.040458 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:43:14.040547 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.040553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.040901 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.041075 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.041350 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.042281 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.043458 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.044238 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.044371 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.044510 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.044656 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.044747 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":141,"threads_started":1}
I20260807 22:43:14.045784 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.040486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:45533 (local address 127.0.0.1:51012)
0807 22:43:14.041302 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.041315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.041409 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.042035 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.042038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.042051 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.042116 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.042120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.043444 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.043448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.044373 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.044383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.045394 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.045398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.045486 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.045521 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.045593 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.045658 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":76,"threads_started":1}
I20260807 22:43:14.045876 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.040683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.26.189.193:45533)
0807 22:43:14.041079 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.041091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.041404 (+ 313us) server_negotiation.cc:407] Connection header received
0807 22:43:14.041505 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.041510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.041685 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.041832 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.042217 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.043321 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.044505 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.045268 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.045400 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.045516 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.045615 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.045696 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":103,"threads_started":1}
I20260807 22:43:14.046401 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.039751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44751 (local address 127.0.0.1:60966)
0807 22:43:14.040784 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.040797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.040879 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.041818 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.041821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.041840 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.041924 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.041929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.043249 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.043253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.044422 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.044433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.046051 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.046055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.046152 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.046169 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.046214 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.046261 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":15,"thread_start_us":96,"threads_started":1}
I20260807 22:43:14.047057 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.040373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.26.189.194:44751)
0807 22:43:14.041394 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.041406 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.041425 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:43:14.041495 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.041500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.041685 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.041832 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.042024 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.043131 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.044556 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.045640 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.045757 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.046732 (+ 975us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.046780 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.046860 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":97,"threads_started":1}
I20260807 22:43:14.048002 7557 tablet_service.cc:1467] Processing CreateTablet for tablet a07584b53a564ec7a7a5ab2e52e592b4 (DEFAULT_TABLE table=test-table [id=c18809365b9a4c598aaa98bdcf53e719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.048420 7122 tablet_service.cc:1467] Processing CreateTablet for tablet a07584b53a564ec7a7a5ab2e52e592b4 (DEFAULT_TABLE table=test-table [id=c18809365b9a4c598aaa98bdcf53e719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.049192 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07584b53a564ec7a7a5ab2e52e592b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.049511 7345 tablet_service.cc:1467] Processing CreateTablet for tablet a07584b53a564ec7a7a5ab2e52e592b4 (DEFAULT_TABLE table=test-table [id=c18809365b9a4c598aaa98bdcf53e719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.050381 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07584b53a564ec7a7a5ab2e52e592b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.050485 7345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07584b53a564ec7a7a5ab2e52e592b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.056653 7696 tablet_bootstrap.cc:492] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Bootstrap starting.
I20260807 22:43:14.057649 7702 tablet_bootstrap.cc:492] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Bootstrap starting.
I20260807 22:43:14.058194 7700 tablet_bootstrap.cc:492] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Bootstrap starting.
I20260807 22:43:14.059280 7696 tablet_bootstrap.cc:654] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.059775 7702 tablet_bootstrap.cc:654] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.060163 7696 log.cc:824] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:14.060333 7700 tablet_bootstrap.cc:654] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.060576 7702 log.cc:824] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:14.061131 7700 log.cc:824] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:14.062127 7696 tablet_bootstrap.cc:492] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: No bootstrap required, opened a new log
I20260807 22:43:14.062332 7696 ts_tablet_manager.cc:1397] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:43:14.062449 7702 tablet_bootstrap.cc:492] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: No bootstrap required, opened a new log
I20260807 22:43:14.062640 7702 ts_tablet_manager.cc:1397] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:43:14.062986 7700 tablet_bootstrap.cc:492] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: No bootstrap required, opened a new log
I20260807 22:43:14.063171 7700 ts_tablet_manager.cc:1397] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:14.065652 7696 raft_consensus.cc:348] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.065857 7696 raft_consensus.cc:374] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.065908 7696 raft_consensus.cc:729] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9703e6c74d4d49b6ac3e861fe54c40, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.066332 7696 consensus_queue.cc:260] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.066354 7702 raft_consensus.cc:348] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.066598 7702 raft_consensus.cc:374] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.066668 7702 raft_consensus.cc:729] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd306189caa2489484baaffe9380186b, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.066859 7700 raft_consensus.cc:348] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.067085 7700 raft_consensus.cc:374] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.067057 7702 consensus_queue.cc:260] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.067147 7700 raft_consensus.cc:729] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe5a1e07ca643cd9540fa220338bdd2, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.067253 7696 ts_tablet_manager.cc:1428] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:14.067591 7220 heartbeater.cc:503] Master 127.26.189.254:44769 was elected leader, sending a full tablet report...
I20260807 22:43:14.067545 7700 consensus_queue.cc:260] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.067955 7702 ts_tablet_manager.cc:1428] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:43:14.068200 7650 heartbeater.cc:503] Master 127.26.189.254:44769 was elected leader, sending a full tablet report...
I20260807 22:43:14.068267 7455 heartbeater.cc:503] Master 127.26.189.254:44769 was elected leader, sending a full tablet report...
I20260807 22:43:14.068403 7700 ts_tablet_manager.cc:1428] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 22:43:14.069968 7221 tablet.cc:2369] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:14.118045 7710 raft_consensus.cc:482] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:14.118335 7710 raft_consensus.cc:504] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.119546 7710 leader_election.cc:288] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695), 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533)
I20260807 22:43:14.124099 7712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.120069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:45533 (local address 127.26.189.194:36047)
0807 22:43:14.120382 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.120394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.120483 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.120699 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.120701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.120715 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.120806 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.120810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.121670 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.121674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.123008 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.123019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.123678 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.123681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.123785 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.123808 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.123873 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.123942 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260807 22:43:14.124076 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.120198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:36047 (local address 127.26.189.193:45533)
0807 22:43:14.120317 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.120321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.120464 (+ 143us) server_negotiation.cc:407] Connection header received
0807 22:43:14.120559 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.120562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.120609 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.120686 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.120898 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.121581 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.123110 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.123557 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.123660 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.123715 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.123901 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.123969 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:43:14.124269 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.119898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:55445 (local address 127.26.189.195:35695)
0807 22:43:14.120016 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.120021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.120039 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:43:14.120117 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.120120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.120167 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.120247 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.120606 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.121378 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.123341 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.123747 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.123853 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.123902 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.124081 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.124165 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:43:14.124265 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.119798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:35695 (local address 127.26.189.194:55445)
0807 22:43:14.119926 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.119941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.120026 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.120351 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.120354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.120374 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.120496 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.120503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.121485 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.121487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.123226 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.123235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.123855 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.123859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.123966 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.123988 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.124053 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.124123 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":15}
I20260807 22:43:14.124786 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a07584b53a564ec7a7a5ab2e52e592b4" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" is_pre_election: true
I20260807 22:43:14.124893 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a07584b53a564ec7a7a5ab2e52e592b4" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" is_pre_election: true
I20260807 22:43:14.125074 7150 raft_consensus.cc:2454] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 0.
I20260807 22:43:14.125165 7581 raft_consensus.cc:2454] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 0.
I20260807 22:43:14.125538 7269 leader_election.cc:302] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9703e6c74d4d49b6ac3e861fe54c40, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.125789 7710 raft_consensus.cc:2781] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:14.125883 7710 raft_consensus.cc:482] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:14.125934 7710 raft_consensus.cc:3037] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.127015 7710 raft_consensus.cc:504] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.127301 7710 leader_election.cc:288] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 election: Requested vote from peers 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695), 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533)
I20260807 22:43:14.127594 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a07584b53a564ec7a7a5ab2e52e592b4" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
I20260807 22:43:14.127705 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a07584b53a564ec7a7a5ab2e52e592b4" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
I20260807 22:43:14.127714 7581 raft_consensus.cc:3037] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.127835 7150 raft_consensus.cc:3037] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.128719 7581 raft_consensus.cc:2454] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 1.
I20260807 22:43:14.128851 7150 raft_consensus.cc:2454] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 1.
I20260807 22:43:14.129001 7269 leader_election.cc:302] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.129153 7710 raft_consensus.cc:2781] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:14.129351 7710 raft_consensus.cc:686] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 1 LEADER]: Becoming Leader. State: Replica: fd306189caa2489484baaffe9380186b, State: Running, Role: LEADER
I20260807 22:43:14.129673 7710 consensus_queue.cc:237] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } }
I20260807 22:43:14.132614 6900 catalog_manager.cc:5495] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b reported cstate change: term changed from 0 to 1, leader changed from <none> to fd306189caa2489484baaffe9380186b (127.26.189.194). New cstate: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:14.144559 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.141329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.26.189.194:44751)
0807 22:43:14.141443 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.141448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.141461 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:14.141607 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.141610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.141659 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.141741 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.141958 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.142482 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.143293 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.143503 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.143585 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.143829 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.144211 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.144313 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.144390 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.144441 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:43:14.149705 7581 raft_consensus.cc:1261] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Refusing update from remote peer fd306189caa2489484baaffe9380186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.149749 7150 raft_consensus.cc:1261] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Refusing update from remote peer fd306189caa2489484baaffe9380186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.150377 7710 consensus_queue.cc:1035] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.150557 7713 consensus_queue.cc:1035] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.158115 7722 mvcc.cc:205] Tried to move back new op lower bound from 7316040065631006720 to 7316040065557700608. Current Snapshot: MvccSnapshot[applied={T|T < 7316040065631006720}]
I20260807 22:43:14.158229 7718 mvcc.cc:205] Tried to move back new op lower bound from 7316040065631006720 to 7316040065557700608. Current Snapshot: MvccSnapshot[applied={T|T < 7316040065631006720}]
I20260807 22:43:14.165723 6901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.165866 6901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.168045 6901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Sending DeleteTablet for 3 replicas of tablet a07584b53a564ec7a7a5ab2e52e592b4
I20260807 22:43:14.168756 7345 tablet_service.cc:1514] Processing DeleteTablet for tablet a07584b53a564ec7a7a5ab2e52e592b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60966
I20260807 22:43:14.169123 7557 tablet_service.cc:1514] Processing DeleteTablet for tablet a07584b53a564ec7a7a5ab2e52e592b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53294
I20260807 22:43:14.169212 7731 tablet_replica.cc:330] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: stopping tablet replica
I20260807 22:43:14.169212 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet a07584b53a564ec7a7a5ab2e52e592b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51012
I20260807 22:43:14.169490 7731 raft_consensus.cc:2229] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:14.169734 7735 tablet_replica.cc:330] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: stopping tablet replica
I20260807 22:43:14.169865 7731 raft_consensus.cc:2258] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.169906 6901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:43:14.170020 7735 raft_consensus.cc:2229] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 22:43:14.170248 6901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:14.170317 7735 raft_consensus.cc:2258] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.170665 7736 tablet_replica.cc:330] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: stopping tablet replica
I20260807 22:43:14.170933 7736 raft_consensus.cc:2229] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.171216 7736 raft_consensus.cc:2258] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.172683 7735 ts_tablet_manager.cc:1905] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.172683 7736 ts_tablet_manager.cc:1905] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.175469 7735 ts_tablet_manager.cc:1918] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:14.175534 7736 ts_tablet_manager.cc:1918] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:14.175627 7735 log.cc:1196] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/a07584b53a564ec7a7a5ab2e52e592b4
I20260807 22:43:14.175633 7736 log.cc:1196] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/a07584b53a564ec7a7a5ab2e52e592b4
I20260807 22:43:14.175944 7736 ts_tablet_manager.cc:1939] T a07584b53a564ec7a7a5ab2e52e592b4 P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting consensus metadata
I20260807 22:43:14.175957 7735 ts_tablet_manager.cc:1939] T a07584b53a564ec7a7a5ab2e52e592b4 P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting consensus metadata
I20260807 22:43:14.176637 7345 tablet_service.cc:1467] Processing CreateTablet for tablet 07177d2d38b143788d80df304cb3b143 (DEFAULT_TABLE table=test-table [id=947c0795c57e4f548bf2358fc8732861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.176910 7345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07177d2d38b143788d80df304cb3b143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.177419 7731 ts_tablet_manager.cc:1905] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.178764 6887 catalog_manager.cc:4841] TS 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695): tablet a07584b53a564ec7a7a5ab2e52e592b4 (table test-table [id=c18809365b9a4c598aaa98bdcf53e719]) successfully deleted
I20260807 22:43:14.178961 6887 catalog_manager.cc:4841] TS 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533): tablet a07584b53a564ec7a7a5ab2e52e592b4 (table test-table [id=c18809365b9a4c598aaa98bdcf53e719]) successfully deleted
I20260807 22:43:14.179621 7557 tablet_service.cc:1467] Processing CreateTablet for tablet 07177d2d38b143788d80df304cb3b143 (DEFAULT_TABLE table=test-table [id=947c0795c57e4f548bf2358fc8732861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.179625 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 07177d2d38b143788d80df304cb3b143 (DEFAULT_TABLE table=test-table [id=947c0795c57e4f548bf2358fc8732861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.179852 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07177d2d38b143788d80df304cb3b143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.179852 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07177d2d38b143788d80df304cb3b143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.179924 7731 ts_tablet_manager.cc:1918] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:14.180018 7731 log.cc:1196] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/a07584b53a564ec7a7a5ab2e52e592b4
I20260807 22:43:14.180305 7731 ts_tablet_manager.cc:1939] T a07584b53a564ec7a7a5ab2e52e592b4 P fd306189caa2489484baaffe9380186b: Deleting consensus metadata
I20260807 22:43:14.181905 7700 tablet_bootstrap.cc:492] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Bootstrap starting.
I20260807 22:43:14.181905 7696 tablet_bootstrap.cc:492] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Bootstrap starting.
I20260807 22:43:14.182145 7702 tablet_bootstrap.cc:492] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Bootstrap starting.
I20260807 22:43:14.182674 6879 catalog_manager.cc:4841] TS fd306189caa2489484baaffe9380186b (127.26.189.194:44751): tablet a07584b53a564ec7a7a5ab2e52e592b4 (table test-table [id=c18809365b9a4c598aaa98bdcf53e719]) successfully deleted
I20260807 22:43:14.182852 7700 tablet_bootstrap.cc:654] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.182852 7696 tablet_bootstrap.cc:654] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.183099 7702 tablet_bootstrap.cc:654] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.184219 7700 tablet_bootstrap.cc:492] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: No bootstrap required, opened a new log
I20260807 22:43:14.184219 7696 tablet_bootstrap.cc:492] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: No bootstrap required, opened a new log
I20260807 22:43:14.184305 7700 ts_tablet_manager.cc:1397] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:14.184305 7696 ts_tablet_manager.cc:1397] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:14.184304 7702 tablet_bootstrap.cc:492] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: No bootstrap required, opened a new log
I20260807 22:43:14.184386 7702 ts_tablet_manager.cc:1397] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:14.184733 7696 raft_consensus.cc:348] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.184733 7700 raft_consensus.cc:348] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.184846 7700 raft_consensus.cc:374] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.184845 7696 raft_consensus.cc:374] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.184885 7696 raft_consensus.cc:729] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9703e6c74d4d49b6ac3e861fe54c40, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.184885 7700 raft_consensus.cc:729] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe5a1e07ca643cd9540fa220338bdd2, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.184830 7702 raft_consensus.cc:348] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.184933 7702 raft_consensus.cc:374] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.184970 7702 raft_consensus.cc:729] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd306189caa2489484baaffe9380186b, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.184996 7700 consensus_queue.cc:260] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.184999 7696 consensus_queue.cc:260] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.185081 7702 consensus_queue.cc:260] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.185266 7700 ts_tablet_manager.cc:1428] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.185266 7696 ts_tablet_manager.cc:1428] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.185345 7702 ts_tablet_manager.cc:1428] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 22:43:14.236171 7651 tablet.cc:2369] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:14.275693 7456 tablet.cc:2369] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:14.319268 7709 raft_consensus.cc:482] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:14.319430 7709 raft_consensus.cc:504] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.320477 7709 leader_election.cc:288] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd306189caa2489484baaffe9380186b (127.26.189.194:44751), 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:14.324888 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.320939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:52573 (local address 127.26.189.195:35695)
0807 22:43:14.321079 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.321083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.321451 (+ 368us) server_negotiation.cc:407] Connection header received
0807 22:43:14.321603 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.321606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.321652 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.321728 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.322006 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.322744 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.323878 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.324371 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.324481 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.324546 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.324693 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.324768 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:43:14.324887 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.320748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:35695 (local address 127.26.189.193:52573)
0807 22:43:14.321353 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.321365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.321466 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.321749 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.321751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.321767 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.321895 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.321900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.322834 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.322838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.323762 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.323772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.324500 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.324503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.324580 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.324604 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.324661 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.324730 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":260,"thread_start_us":104,"threads_started":1}
I20260807 22:43:14.325194 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.320787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37813 (local address 127.26.189.194:44751)
0807 22:43:14.320908 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.320912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.321203 (+ 291us) server_negotiation.cc:407] Connection header received
0807 22:43:14.321346 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.321349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.321390 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.321451 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.321849 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.322636 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.324125 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.324655 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.324774 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.324818 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.325026 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.325103 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:43:14.325342 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07177d2d38b143788d80df304cb3b143" candidate_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" is_pre_election: true
I20260807 22:43:14.325505 7581 raft_consensus.cc:2454] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9703e6c74d4d49b6ac3e861fe54c40 in term 0.
I20260807 22:43:14.325546 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.320745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44751 (local address 127.26.189.193:37813)
0807 22:43:14.321096 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.321110 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.321239 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.321535 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.321537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.321572 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.321706 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.321713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.322738 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.322742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.323997 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.324008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.324779 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.324783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.324872 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.324895 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.325338 (+ 443us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.325415 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260807 22:43:14.325861 7049 leader_election.cc:302] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, 4f9703e6c74d4d49b6ac3e861fe54c40; no voters:
I20260807 22:43:14.326114 7709 raft_consensus.cc:2781] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:14.326179 7709 raft_consensus.cc:482] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:14.326113 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07177d2d38b143788d80df304cb3b143" candidate_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd306189caa2489484baaffe9380186b" is_pre_election: true
I20260807 22:43:14.326215 7709 raft_consensus.cc:3037] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.326356 7379 raft_consensus.cc:2454] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f9703e6c74d4d49b6ac3e861fe54c40 in term 0.
I20260807 22:43:14.327353 7709 raft_consensus.cc:504] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.327666 7709 leader_election.cc:288] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [CANDIDATE]: Term 1 election: Requested vote from peers fd306189caa2489484baaffe9380186b (127.26.189.194:44751), 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:14.327968 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07177d2d38b143788d80df304cb3b143" candidate_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
I20260807 22:43:14.328078 7581 raft_consensus.cc:3037] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.328923 7581 raft_consensus.cc:2454] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9703e6c74d4d49b6ac3e861fe54c40 in term 1.
I20260807 22:43:14.329152 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07177d2d38b143788d80df304cb3b143" candidate_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd306189caa2489484baaffe9380186b"
I20260807 22:43:14.329206 7049 leader_election.cc:302] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, 4f9703e6c74d4d49b6ac3e861fe54c40; no voters:
I20260807 22:43:14.329286 7379 raft_consensus.cc:3037] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.329366 7709 raft_consensus.cc:2781] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:14.329589 7709 raft_consensus.cc:686] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9703e6c74d4d49b6ac3e861fe54c40, State: Running, Role: LEADER
I20260807 22:43:14.329826 7709 consensus_queue.cc:237] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.330271 7379 raft_consensus.cc:2454] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f9703e6c74d4d49b6ac3e861fe54c40 in term 1.
I20260807 22:43:14.331960 6901 catalog_manager.cc:5495] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142593, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.254@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:14.363452 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.355945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.26.189.254:44769)
0807 22:43:14.356071 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.356077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.356243 (+ 166us) server_negotiation.cc:407] Connection header received
0807 22:43:14.356360 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.356362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.356401 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.356463 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:14.357093 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.357612 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.358392 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.358601 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.358691 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.360786 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:14.360790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:14.360792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:14.360812 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:14.361874 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:14.362287 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:14.362322 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:14.362324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:14.362398 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:14.362636 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:14.362640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:14.362642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:14.362806 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:14.362888 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.363160 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.363310 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":47}
I20260807 22:43:14.363454 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.355892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44769 (local address 127.0.0.1:53506)
0807 22:43:14.356159 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.356171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.356277 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.356493 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.356495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.356885 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:14.356978 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.356985 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.357708 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.357711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.358272 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.358281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.358701 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.358704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.358828 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.359339 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:14.359355 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:14.360670 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:14.361969 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:14.361978 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:14.361984 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:14.362201 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:14.362477 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:14.362480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:14.362482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:14.362555 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:14.362902 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:14.362908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:14.363009 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.363100 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.363260 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":31,"thread_start_us":98,"threads_started":1}
I20260807 22:43:14.366882 6900 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53506:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:43:14.367570 6900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:14.371928 7122 tablet_service.cc:1467] Processing CreateTablet for tablet f29f9cf564794ddc98d3821b6efbc555 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91256b7709544b2888b97f58517142ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:14.372030 7345 tablet_service.cc:1467] Processing CreateTablet for tablet f29f9cf564794ddc98d3821b6efbc555 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91256b7709544b2888b97f58517142ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:14.372099 7557 tablet_service.cc:1467] Processing CreateTablet for tablet f29f9cf564794ddc98d3821b6efbc555 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91256b7709544b2888b97f58517142ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:43:14.372344 7345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f29f9cf564794ddc98d3821b6efbc555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.372344 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f29f9cf564794ddc98d3821b6efbc555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.372496 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f29f9cf564794ddc98d3821b6efbc555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.374913 7700 tablet_bootstrap.cc:492] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: Bootstrap starting.
I20260807 22:43:14.375073 7696 tablet_bootstrap.cc:492] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: Bootstrap starting.
I20260807 22:43:14.375077 7702 tablet_bootstrap.cc:492] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: Bootstrap starting.
I20260807 22:43:14.375881 7696 tablet_bootstrap.cc:654] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.375913 7702 tablet_bootstrap.cc:654] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.375913 7700 tablet_bootstrap.cc:654] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.376986 7696 tablet_bootstrap.cc:492] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: No bootstrap required, opened a new log
I20260807 22:43:14.377068 7696 ts_tablet_manager.cc:1397] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:14.377321 7702 tablet_bootstrap.cc:492] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: No bootstrap required, opened a new log
I20260807 22:43:14.377390 7702 ts_tablet_manager.cc:1397] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:14.377449 7696 raft_consensus.cc:348] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.377547 7696 raft_consensus.cc:374] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.377584 7696 raft_consensus.cc:729] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9703e6c74d4d49b6ac3e861fe54c40, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.377713 7696 consensus_queue.cc:260] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.377751 7702 raft_consensus.cc:348] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.377846 7702 raft_consensus.cc:374] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.377882 7702 raft_consensus.cc:729] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd306189caa2489484baaffe9380186b, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.377966 7709 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.378000 7702 consensus_queue.cc:260] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.378106 7709 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.378264 7702 ts_tablet_manager.cc:1428] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:43:14.378684 7710 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.378814 7710 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.377971 7696 ts_tablet_manager.cc:1428] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:43:14.379149 7700 tablet_bootstrap.cc:492] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: No bootstrap required, opened a new log
I20260807 22:43:14.379251 7700 ts_tablet_manager.cc:1397] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:43:14.379719 7700 raft_consensus.cc:348] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.379835 7700 raft_consensus.cc:374] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.379873 7700 raft_consensus.cc:729] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe5a1e07ca643cd9540fa220338bdd2, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.379990 7700 consensus_queue.cc:260] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.380240 7700 ts_tablet_manager.cc:1428] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:43:14.380244 7711 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.380400 7711 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.406348 7710 raft_consensus.cc:482] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:14.406484 7710 raft_consensus.cc:504] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.406834 7710 leader_election.cc:288] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:14.407246 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f29f9cf564794ddc98d3821b6efbc555" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" is_pre_election: true
I20260807 22:43:14.407305 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f29f9cf564794ddc98d3821b6efbc555" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" is_pre_election: true
I20260807 22:43:14.407382 7150 raft_consensus.cc:2454] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 0.
I20260807 22:43:14.407470 7581 raft_consensus.cc:2454] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 0.
I20260807 22:43:14.407670 7269 leader_election.cc:302] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9703e6c74d4d49b6ac3e861fe54c40, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.407855 7710 raft_consensus.cc:2781] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:14.407914 7710 raft_consensus.cc:482] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:14.407958 7710 raft_consensus.cc:3037] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.408926 7710 raft_consensus.cc:504] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.409235 7710 leader_election.cc:288] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 election: Requested vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:14.409551 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f29f9cf564794ddc98d3821b6efbc555" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
I20260807 22:43:14.409577 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f29f9cf564794ddc98d3821b6efbc555" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
I20260807 22:43:14.409680 7581 raft_consensus.cc:3037] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.409677 7150 raft_consensus.cc:3037] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.410607 7581 raft_consensus.cc:2454] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 1.
I20260807 22:43:14.410663 7150 raft_consensus.cc:2454] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 1.
I20260807 22:43:14.410897 7269 leader_election.cc:302] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.411067 7710 raft_consensus.cc:2781] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:14.411166 7710 raft_consensus.cc:686] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [term 1 LEADER]: Becoming Leader. State: Replica: fd306189caa2489484baaffe9380186b, State: Running, Role: LEADER
I20260807 22:43:14.411322 7710 consensus_queue.cc:237] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.411953 7713 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: TxnStatusTablet state changed. Reason: New leader fd306189caa2489484baaffe9380186b. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.412063 7713 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:14.412317 7766 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:43:14.412482 7766 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:43:14.412973 6899 catalog_manager.cc:5495] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b reported cstate change: term changed from 0 to 1, leader changed from <none> to fd306189caa2489484baaffe9380186b (127.26.189.194). New cstate: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:14.433420 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.429993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44751 (local address 127.0.0.1:60970)
0807 22:43:14.430249 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.430262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.430392 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.430650 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.430652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.430658 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:14.430746 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.430750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.431711 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.431713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.432305 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.432311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.432825 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.432828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.432890 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.432931 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.433195 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.433204 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.433247 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.433313 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260807 22:43:14.433445 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.430122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.26.189.194:44751)
0807 22:43:14.430248 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.430253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.430314 (+ 61us) server_negotiation.cc:407] Connection header received
0807 22:43:14.430495 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.430498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.430549 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.430641 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.430826 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.431585 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.432396 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.432735 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.432834 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.433036 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.433118 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.433180 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.433259 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.433310 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:43:14.435817 7150 raft_consensus.cc:1261] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Refusing update from remote peer fd306189caa2489484baaffe9380186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.435837 7581 raft_consensus.cc:1261] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Refusing update from remote peer fd306189caa2489484baaffe9380186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.436236 7710 consensus_queue.cc:1035] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.436389 7713 consensus_queue.cc:1035] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.437708 7709 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: TxnStatusTablet state changed. Reason: New leader fd306189caa2489484baaffe9380186b. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.437824 7709 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.437829 7711 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: TxnStatusTablet state changed. Reason: New leader fd306189caa2489484baaffe9380186b. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.437937 7711 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.439479 7713 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.439585 7713 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:14.440030 7716 mvcc.cc:205] Tried to move back new op lower bound from 7316040066804989952 to 7316040066710097920. Current Snapshot: MvccSnapshot[applied={T|T < 7316040066804989952 or (T in {7316040066804989952})}]
I20260807 22:43:14.440958 7709 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.441057 7709 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P 4f9703e6c74d4d49b6ac3e861fe54c40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.447525 7713 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.447643 7713 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P fd306189caa2489484baaffe9380186b: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:43:14.448199 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.443958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.26.189.193:45533)
0807 22:43:14.444072 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.444076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.444090 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:43:14.444211 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.444214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.444273 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.444373 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.444539 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.445308 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.446173 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.446533 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.446639 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.446877 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.447428 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.447530 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.447997 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.448052 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:43:14.448567 7711 tablet_replica.cc:439] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } } }
I20260807 22:43:14.448695 7711 tablet_replica.cc:442] T f29f9cf564794ddc98d3821b6efbc555 P 3fe5a1e07ca643cd9540fa220338bdd2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:43:14.458431 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.455009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:45371 (local address 127.26.189.194:44751)
0807 22:43:14.455117 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.455121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.455379 (+ 258us) server_negotiation.cc:407] Connection header received
0807 22:43:14.455491 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.455493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.455533 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.455596 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.455818 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.456269 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.457447 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.457655 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.457725 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.458015 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.458080 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.458140 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.458234 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.458314 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:43:14.458701 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.454910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44751 (local address 127.26.189.193:45371)
0807 22:43:14.455297 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.455311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.455411 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.455615 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.455618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.455625 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:14.455742 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.455747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.456352 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.456355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.457348 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.457359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.457741 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.457744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.457837 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.457885 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.458158 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.458163 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.458223 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.458318 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":101,"threads_started":1}
I20260807 22:43:14.465049 7581 raft_consensus.cc:1261] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Refusing update from remote peer 4f9703e6c74d4d49b6ac3e861fe54c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.465245 7379 raft_consensus.cc:1261] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Refusing update from remote peer 4f9703e6c74d4d49b6ac3e861fe54c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.465758 7744 consensus_queue.cc:1035] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.465945 7709 consensus_queue.cc:1035] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.490996 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.487121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:43869 (local address 127.26.189.193:45533)
0807 22:43:14.487230 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.487235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.487486 (+ 251us) server_negotiation.cc:407] Connection header received
0807 22:43:14.487570 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.487573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.487629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.487701 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.487853 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.488620 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.490005 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.490226 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.490332 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.490627 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.490707 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.490762 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.490853 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.490897 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:43:14.491299 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.487055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:45533 (local address 127.26.189.194:43869)
0807 22:43:14.487401 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.487413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.487501 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.487702 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.487704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.487709 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:14.487793 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.487797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.488744 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.488747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.489902 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.489915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.490351 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.490355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.490442 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.490495 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.490778 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.490784 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.490846 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.490922 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260807 22:43:14.511065 6901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.511219 6901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.513134 6901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Sending DeleteTablet for 3 replicas of tablet 07177d2d38b143788d80df304cb3b143
I20260807 22:43:14.513707 7345 tablet_service.cc:1514] Processing DeleteTablet for tablet 07177d2d38b143788d80df304cb3b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60966
I20260807 22:43:14.513708 7557 tablet_service.cc:1514] Processing DeleteTablet for tablet 07177d2d38b143788d80df304cb3b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53294
I20260807 22:43:14.513875 7735 tablet_replica.cc:330] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: stopping tablet replica
I20260807 22:43:14.513890 7731 tablet_replica.cc:330] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: stopping tablet replica
I20260807 22:43:14.513970 7735 raft_consensus.cc:2229] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.514019 7731 raft_consensus.cc:2229] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.514051 7735 raft_consensus.cc:2258] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.514137 7731 raft_consensus.cc:2258] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.514854 7731 ts_tablet_manager.cc:1905] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.514854 7735 ts_tablet_manager.cc:1905] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.517520 7731 ts_tablet_manager.cc:1918] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:14.517519 7735 ts_tablet_manager.cc:1918] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:14.517587 7731 log.cc:1196] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/07177d2d38b143788d80df304cb3b143
I20260807 22:43:14.517598 7735 log.cc:1196] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/07177d2d38b143788d80df304cb3b143
I20260807 22:43:14.517828 7735 ts_tablet_manager.cc:1939] T 07177d2d38b143788d80df304cb3b143 P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting consensus metadata
I20260807 22:43:14.517828 7731 ts_tablet_manager.cc:1939] T 07177d2d38b143788d80df304cb3b143 P fd306189caa2489484baaffe9380186b: Deleting consensus metadata
I20260807 22:43:14.518456 6879 catalog_manager.cc:4841] TS fd306189caa2489484baaffe9380186b (127.26.189.194:44751): tablet 07177d2d38b143788d80df304cb3b143 (table test-table [id=947c0795c57e4f548bf2358fc8732861]) successfully deleted
I20260807 22:43:14.518698 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 07177d2d38b143788d80df304cb3b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51012
I20260807 22:43:14.518874 7736 tablet_replica.cc:330] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: stopping tablet replica
I20260807 22:43:14.519002 7736 raft_consensus.cc:2229] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:14.519102 6887 catalog_manager.cc:4841] TS 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695): tablet 07177d2d38b143788d80df304cb3b143 (table test-table [id=947c0795c57e4f548bf2358fc8732861]) successfully deleted
I20260807 22:43:14.519290 7736 raft_consensus.cc:2258] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.520018 7736 ts_tablet_manager.cc:1905] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.522614 7736 ts_tablet_manager.cc:1918] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:14.522676 7736 log.cc:1196] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/07177d2d38b143788d80df304cb3b143
I20260807 22:43:14.522895 7736 ts_tablet_manager.cc:1939] T 07177d2d38b143788d80df304cb3b143 P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting consensus metadata
I20260807 22:43:14.523541 6887 catalog_manager.cc:4841] TS 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533): tablet 07177d2d38b143788d80df304cb3b143 (table test-table [id=947c0795c57e4f548bf2358fc8732861]) successfully deleted
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:43:14.544720 6973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:14.538952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.26.189.254:44769)
0807 22:43:14.539075 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.539080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.539319 (+ 239us) server_negotiation.cc:407] Connection header received
0807 22:43:14.539445 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.539449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.539497 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.539561 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:14.540242 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.540769 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.541599 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.541823 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.541909 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.544626 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53512: BlockingRecv error: recv got EOF from 127.0.0.1:53512 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 22:43:14 dist-test-slave-4tdj krb5kdc[6831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM
I20260807 22:43:14.554483 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.546514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.26.189.254:44769)
0807 22:43:14.546652 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.546657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.547158 (+ 501us) server_negotiation.cc:407] Connection header received
0807 22:43:14.547260 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.547263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.547302 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.547363 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:14.547910 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.548384 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.549269 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.549473 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.549555 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.551700 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:14.551703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:14.551706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:14.551730 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:14.552958 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:14.553329 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:14.553332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:14.553334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:14.553407 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:14.553662 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:14.553665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:14.553667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:14.553832 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:14.553946 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.554172 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.554273 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:43:14.556805 6901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:14.557240 6901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:14.561810 7122 tablet_service.cc:1467] Processing CreateTablet for tablet aa40b2b7f9bd49d08b990e433b9920fe (DEFAULT_TABLE table=test-table [id=5d8cc643ac6b43e48ac4f70465dcc3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.562001 7345 tablet_service.cc:1467] Processing CreateTablet for tablet aa40b2b7f9bd49d08b990e433b9920fe (DEFAULT_TABLE table=test-table [id=5d8cc643ac6b43e48ac4f70465dcc3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.562114 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa40b2b7f9bd49d08b990e433b9920fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.562260 7345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa40b2b7f9bd49d08b990e433b9920fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.564525 7702 tablet_bootstrap.cc:492] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Bootstrap starting.
I20260807 22:43:14.564525 7696 tablet_bootstrap.cc:492] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Bootstrap starting.
I20260807 22:43:14.565412 7696 tablet_bootstrap.cc:654] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.565412 7702 tablet_bootstrap.cc:654] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.566643 7696 tablet_bootstrap.cc:492] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: No bootstrap required, opened a new log
I20260807 22:43:14.566730 7696 ts_tablet_manager.cc:1397] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:14.567178 7696 raft_consensus.cc:348] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.567288 7696 raft_consensus.cc:374] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.567339 7696 raft_consensus.cc:729] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9703e6c74d4d49b6ac3e861fe54c40, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.567492 7702 tablet_bootstrap.cc:492] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: No bootstrap required, opened a new log
I20260807 22:43:14.567459 7696 consensus_queue.cc:260] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.567564 7702 ts_tablet_manager.cc:1397] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:43:14.567727 7696 ts_tablet_manager.cc:1428] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.567934 7702 raft_consensus.cc:348] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.568056 7702 raft_consensus.cc:374] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.568099 7702 raft_consensus.cc:729] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd306189caa2489484baaffe9380186b, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.568205 7702 consensus_queue.cc:260] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.568434 7702 ts_tablet_manager.cc:1428] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.570580 7557 tablet_service.cc:1467] Processing CreateTablet for tablet aa40b2b7f9bd49d08b990e433b9920fe (DEFAULT_TABLE table=test-table [id=5d8cc643ac6b43e48ac4f70465dcc3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.570890 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa40b2b7f9bd49d08b990e433b9920fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.572901 7700 tablet_bootstrap.cc:492] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Bootstrap starting.
I20260807 22:43:14.573804 7700 tablet_bootstrap.cc:654] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.577749 7700 tablet_bootstrap.cc:492] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: No bootstrap required, opened a new log
I20260807 22:43:14.577826 7700 ts_tablet_manager.cc:1397] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 22:43:14.578562 7700 raft_consensus.cc:348] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.578665 7700 raft_consensus.cc:374] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.578694 7700 raft_consensus.cc:729] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe5a1e07ca643cd9540fa220338bdd2, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.578796 7700 consensus_queue.cc:260] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.579031 7700 ts_tablet_manager.cc:1428] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.584980 7711 raft_consensus.cc:482] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:14.585083 7711 raft_consensus.cc:504] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.586210 7711 leader_election.cc:288] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), fd306189caa2489484baaffe9380186b (127.26.189.194:44751)
I20260807 22:43:14.589704 7772 raft_consensus.cc:482] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:14.589814 7772 raft_consensus.cc:504] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.590164 7772 leader_election.cc:288] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:14.590670 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" is_pre_election: true
I20260807 22:43:14.590804 7150 raft_consensus.cc:2454] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 0.
I20260807 22:43:14.591028 7578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" is_pre_election: true
I20260807 22:43:14.591147 7578 raft_consensus.cc:2454] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 0.
I20260807 22:43:14.591536 7269 leader_election.cc:302] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f9703e6c74d4d49b6ac3e861fe54c40, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.591706 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.586706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:41413 (local address 127.26.189.193:45533)
0807 22:43:14.586805 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.586809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.586967 (+ 158us) server_negotiation.cc:407] Connection header received
0807 22:43:14.587154 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.587157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.587201 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.587245 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.587646 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.588420 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.590355 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.590874 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.590958 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.591010 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.591529 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.591604 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:43:14.591833 7772 raft_consensus.cc:2781] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:14.591895 7772 raft_consensus.cc:482] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:14.591940 7772 raft_consensus.cc:3037] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.592115 7808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.586406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:45533 (local address 127.26.189.195:41413)
0807 22:43:14.586888 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.586901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.587033 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.587377 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.587380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.587396 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.587522 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.587528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.588701 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.588704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.590144 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.590156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.591302 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.591306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.591393 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.591418 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.591873 (+ 455us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.591955 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"spinlock_wait_cycles":21376,"thread_start_us":72,"threads_started":1}
I20260807 22:43:14.592449 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" is_pre_election: true
I20260807 22:43:14.592561 7150 raft_consensus.cc:2454] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fe5a1e07ca643cd9540fa220338bdd2 in term 0.
I20260807 22:43:14.592861 7492 leader_election.cc:302] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, 4f9703e6c74d4d49b6ac3e861fe54c40; no voters:
I20260807 22:43:14.593146 7772 raft_consensus.cc:504] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.593236 7711 raft_consensus.cc:2781] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:14.593286 7711 raft_consensus.cc:482] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:14.593319 7711 raft_consensus.cc:3037] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.593453 7772 leader_election.cc:288] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 election: Requested vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695)
I20260807 22:43:14.593982 7578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe5a1e07ca643cd9540fa220338bdd2"
I20260807 22:43:14.594188 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "fd306189caa2489484baaffe9380186b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
I20260807 22:43:14.594277 7150 raft_consensus.cc:3037] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.594395 7711 raft_consensus.cc:504] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.594702 7711 leader_election.cc:288] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), fd306189caa2489484baaffe9380186b (127.26.189.194:44751)
I20260807 22:43:14.595314 7150 raft_consensus.cc:2454] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd306189caa2489484baaffe9380186b in term 1.
I20260807 22:43:14.595624 7578 raft_consensus.cc:2379] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fd306189caa2489484baaffe9380186b in current term 1: Already voted for candidate 3fe5a1e07ca643cd9540fa220338bdd2 in this term.
I20260807 22:43:14.596166 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.586597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:33057 (local address 127.26.189.194:44751)
0807 22:43:14.586741 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.586746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.587023 (+ 277us) server_negotiation.cc:407] Connection header received
0807 22:43:14.587202 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.587205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.587252 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.587292 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:14.587689 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.588463 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.591387 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.593265 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.593379 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.593424 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.595991 (+ 2567us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.596069 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:43:14.596190 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.586538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44751 (local address 127.26.189.195:33057)
0807 22:43:14.586948 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.586961 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.587098 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.587407 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.587410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.587447 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:14.587566 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.587573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.589184 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.589187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.591257 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.591268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.595764 (+ 4496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.595768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.595867 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.595886 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.595957 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.596023 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":3,"spinlock_wait_cycles":2688,"thread_start_us":86,"threads_started":1}
I20260807 22:43:14.596375 7269 leader_election.cc:302] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f9703e6c74d4d49b6ac3e861fe54c40, fd306189caa2489484baaffe9380186b; no voters: 3fe5a1e07ca643cd9540fa220338bdd2
I20260807 22:43:14.596740 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd306189caa2489484baaffe9380186b" is_pre_election: true
I20260807 22:43:14.596788 7772 raft_consensus.cc:2781] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:14.596838 7378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd306189caa2489484baaffe9380186b"
I20260807 22:43:14.596871 7772 raft_consensus.cc:686] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 1 LEADER]: Becoming Leader. State: Replica: fd306189caa2489484baaffe9380186b, State: Running, Role: LEADER
I20260807 22:43:14.596995 7772 consensus_queue.cc:237] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.598271 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa40b2b7f9bd49d08b990e433b9920fe" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
I20260807 22:43:14.598448 7150 raft_consensus.cc:2379] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fe5a1e07ca643cd9540fa220338bdd2 in current term 1: Already voted for candidate fd306189caa2489484baaffe9380186b in this term.
I20260807 22:43:14.598737 7492 leader_election.cc:302] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2; no voters: 4f9703e6c74d4d49b6ac3e861fe54c40, fd306189caa2489484baaffe9380186b
I20260807 22:43:14.598757 6901 catalog_manager.cc:5495] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b reported cstate change: term changed from 0 to 1, leader changed from <none> to fd306189caa2489484baaffe9380186b (127.26.189.194). New cstate: current_term: 1 leader_uuid: "fd306189caa2489484baaffe9380186b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:14.598915 7711 raft_consensus.cc:2726] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 22:43:14.607179 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.604109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.26.189.194:44751)
0807 22:43:14.604224 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.604228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.604244 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:43:14.604347 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.604350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.604395 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.604468 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.604667 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.605412 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.606170 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.606523 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.606624 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.606819 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.606894 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.606957 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.607011 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.607059 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:43:14.608829 7578 raft_consensus.cc:1261] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Refusing update from remote peer fd306189caa2489484baaffe9380186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.608919 7150 raft_consensus.cc:1261] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Refusing update from remote peer fd306189caa2489484baaffe9380186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.609280 7772 consensus_queue.cc:1035] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.609416 7713 consensus_queue.cc:1035] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.617110 6900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.617259 6900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.619187 6900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Sending DeleteTablet for 3 replicas of tablet aa40b2b7f9bd49d08b990e433b9920fe
I20260807 22:43:14.619714 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet aa40b2b7f9bd49d08b990e433b9920fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51012
I20260807 22:43:14.619896 7557 tablet_service.cc:1514] Processing DeleteTablet for tablet aa40b2b7f9bd49d08b990e433b9920fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53294
I20260807 22:43:14.620074 7736 tablet_replica.cc:330] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: stopping tablet replica
I20260807 22:43:14.620105 7345 tablet_service.cc:1514] Processing DeleteTablet for tablet aa40b2b7f9bd49d08b990e433b9920fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60966
I20260807 22:43:14.620195 7736 raft_consensus.cc:2229] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.620246 7731 tablet_replica.cc:330] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: stopping tablet replica
I20260807 22:43:14.620318 7736 raft_consensus.cc:2258] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.620352 7731 raft_consensus.cc:2229] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:14.620481 7735 tablet_replica.cc:330] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: stopping tablet replica
I20260807 22:43:14.620545 7731 raft_consensus.cc:2258] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.620606 7735 raft_consensus.cc:2229] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.620718 7735 raft_consensus.cc:2258] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.621407 7731 ts_tablet_manager.cc:1905] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.621407 7735 ts_tablet_manager.cc:1905] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.621645 7736 ts_tablet_manager.cc:1905] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.623713 7731 ts_tablet_manager.cc:1918] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:14.623713 7735 ts_tablet_manager.cc:1918] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:14.623780 7735 log.cc:1196] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/aa40b2b7f9bd49d08b990e433b9920fe
I20260807 22:43:14.623780 7731 log.cc:1196] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/aa40b2b7f9bd49d08b990e433b9920fe
I20260807 22:43:14.623981 7736 ts_tablet_manager.cc:1918] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:14.624022 7735 ts_tablet_manager.cc:1939] T aa40b2b7f9bd49d08b990e433b9920fe P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting consensus metadata
I20260807 22:43:14.624022 7731 ts_tablet_manager.cc:1939] T aa40b2b7f9bd49d08b990e433b9920fe P fd306189caa2489484baaffe9380186b: Deleting consensus metadata
I20260807 22:43:14.624043 7736 log.cc:1196] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/aa40b2b7f9bd49d08b990e433b9920fe
I20260807 22:43:14.624297 7736 ts_tablet_manager.cc:1939] T aa40b2b7f9bd49d08b990e433b9920fe P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting consensus metadata
I20260807 22:43:14.624753 6879 catalog_manager.cc:4841] TS fd306189caa2489484baaffe9380186b (127.26.189.194:44751): tablet aa40b2b7f9bd49d08b990e433b9920fe (table test-table [id=5d8cc643ac6b43e48ac4f70465dcc3cc]) successfully deleted
I20260807 22:43:14.624747 6900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:43:14.625049 6887 catalog_manager.cc:4841] TS 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695): tablet aa40b2b7f9bd49d08b990e433b9920fe (table test-table [id=5d8cc643ac6b43e48ac4f70465dcc3cc]) successfully deleted
W20260807 22:43:14.625114 6900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:14.625684 6887 catalog_manager.cc:4841] TS 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533): tablet aa40b2b7f9bd49d08b990e433b9920fe (table test-table [id=5d8cc643ac6b43e48ac4f70465dcc3cc]) successfully deleted
I20260807 22:43:14.629195 7122 tablet_service.cc:1467] Processing CreateTablet for tablet e800785827d541d7960fe2769ee2589e (DEFAULT_TABLE table=test-table [id=c567e3355c6840b9b902838616e91fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.629434 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e800785827d541d7960fe2769ee2589e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.629421 7345 tablet_service.cc:1467] Processing CreateTablet for tablet e800785827d541d7960fe2769ee2589e (DEFAULT_TABLE table=test-table [id=c567e3355c6840b9b902838616e91fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.629665 7345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e800785827d541d7960fe2769ee2589e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.631460 7702 tablet_bootstrap.cc:492] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Bootstrap starting.
I20260807 22:43:14.631564 7696 tablet_bootstrap.cc:492] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: Bootstrap starting.
I20260807 22:43:14.631623 7557 tablet_service.cc:1467] Processing CreateTablet for tablet e800785827d541d7960fe2769ee2589e (DEFAULT_TABLE table=test-table [id=c567e3355c6840b9b902838616e91fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:14.631847 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e800785827d541d7960fe2769ee2589e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:14.632298 7702 tablet_bootstrap.cc:654] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.632299 7696 tablet_bootstrap.cc:654] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.633901 7700 tablet_bootstrap.cc:492] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Bootstrap starting.
I20260807 22:43:14.634820 7702 tablet_bootstrap.cc:492] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: No bootstrap required, opened a new log
I20260807 22:43:14.634825 7700 tablet_bootstrap.cc:654] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:14.634914 7702 ts_tablet_manager.cc:1397] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:43:14.635286 7702 raft_consensus.cc:348] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.635407 7702 raft_consensus.cc:374] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.635447 7702 raft_consensus.cc:729] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd306189caa2489484baaffe9380186b, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.635560 7702 consensus_queue.cc:260] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.635699 7696 tablet_bootstrap.cc:492] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: No bootstrap required, opened a new log
I20260807 22:43:14.635769 7696 ts_tablet_manager.cc:1397] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:43:14.635840 7702 ts_tablet_manager.cc:1428] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.636133 7696 raft_consensus.cc:348] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.636247 7696 raft_consensus.cc:374] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.636287 7696 raft_consensus.cc:729] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9703e6c74d4d49b6ac3e861fe54c40, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.636409 7696 consensus_queue.cc:260] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.636673 7696 ts_tablet_manager.cc:1428] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:43:14.637917 7700 tablet_bootstrap.cc:492] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: No bootstrap required, opened a new log
I20260807 22:43:14.637992 7700 ts_tablet_manager.cc:1397] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 22:43:14.638391 7700 raft_consensus.cc:348] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.638496 7700 raft_consensus.cc:374] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:14.638530 7700 raft_consensus.cc:729] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe5a1e07ca643cd9540fa220338bdd2, State: Initialized, Role: FOLLOWER
I20260807 22:43:14.638643 7700 consensus_queue.cc:260] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.638867 7700 ts_tablet_manager.cc:1428] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:43:14.873788 7711 raft_consensus.cc:482] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:14.873955 7711 raft_consensus.cc:504] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.874361 7711 leader_election.cc:288] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), fd306189caa2489484baaffe9380186b (127.26.189.194:44751)
I20260807 22:43:14.874822 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e800785827d541d7960fe2769ee2589e" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd306189caa2489484baaffe9380186b" is_pre_election: true
I20260807 22:43:14.874837 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e800785827d541d7960fe2769ee2589e" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" is_pre_election: true
I20260807 22:43:14.874985 7150 raft_consensus.cc:2454] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fe5a1e07ca643cd9540fa220338bdd2 in term 0.
I20260807 22:43:14.874980 7379 raft_consensus.cc:2454] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fe5a1e07ca643cd9540fa220338bdd2 in term 0.
I20260807 22:43:14.875273 7488 leader_election.cc:302] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.875468 7711 raft_consensus.cc:2781] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:14.875528 7711 raft_consensus.cc:482] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:14.875555 7711 raft_consensus.cc:3037] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.876518 7711 raft_consensus.cc:504] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.876845 7711 leader_election.cc:288] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533), fd306189caa2489484baaffe9380186b (127.26.189.194:44751)
I20260807 22:43:14.877182 7150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e800785827d541d7960fe2769ee2589e" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40"
I20260807 22:43:14.877194 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e800785827d541d7960fe2769ee2589e" candidate_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd306189caa2489484baaffe9380186b"
I20260807 22:43:14.877302 7379 raft_consensus.cc:3037] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.877306 7150 raft_consensus.cc:3037] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:14.878177 7150 raft_consensus.cc:2454] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fe5a1e07ca643cd9540fa220338bdd2 in term 1.
I20260807 22:43:14.878192 7379 raft_consensus.cc:2454] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fe5a1e07ca643cd9540fa220338bdd2 in term 1.
I20260807 22:43:14.878451 7488 leader_election.cc:302] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe5a1e07ca643cd9540fa220338bdd2, fd306189caa2489484baaffe9380186b; no voters:
I20260807 22:43:14.878600 7711 raft_consensus.cc:2781] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:14.878821 7711 raft_consensus.cc:686] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 LEADER]: Becoming Leader. State: Replica: 3fe5a1e07ca643cd9540fa220338bdd2, State: Running, Role: LEADER
I20260807 22:43:14.879063 7711 consensus_queue.cc:237] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } }
I20260807 22:43:14.881174 6900 catalog_manager.cc:5495] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195). New cstate: current_term: 1 leader_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fe5a1e07ca643cd9540fa220338bdd2" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35695 } health_report { overall_health: HEALTHY } } }
I20260807 22:43:14.930370 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.927130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.26.189.195:35695)
0807 22:43:14.927379 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.927383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.927394 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:43:14.927464 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.927466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.927517 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.927613 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.927828 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.928355 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.929095 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.929312 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.929377 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.929570 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.930006 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.930086 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.930166 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.930213 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
I20260807 22:43:14.939879 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.937053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:47671 (local address 127.26.189.194:44751)
0807 22:43:14.937187 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.937191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.937308 (+ 117us) server_negotiation.cc:407] Connection header received
0807 22:43:14.937411 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.937413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.937455 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.937514 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.937704 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.938182 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.939007 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.939211 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.939282 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.939521 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.939581 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.939657 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.939725 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.939772 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:43:14.939910 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.936867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44751 (local address 127.26.189.195:47671)
0807 22:43:14.937240 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.937252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.937338 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.937527 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.937529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.937535 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:14.937630 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.937634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.938259 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.938261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.938916 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.938926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.939293 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.939295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.939378 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.939424 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.939667 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.939671 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.939710 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.939787 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":94,"threads_started":1}
I20260807 22:43:14.945323 7150 raft_consensus.cc:1261] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Refusing update from remote peer 3fe5a1e07ca643cd9540fa220338bdd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.945323 7379 raft_consensus.cc:1261] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Refusing update from remote peer 3fe5a1e07ca643cd9540fa220338bdd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:14.945791 7819 consensus_queue.cc:1035] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9703e6c74d4d49b6ac3e861fe54c40" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.945955 7711 consensus_queue.cc:1035] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd306189caa2489484baaffe9380186b" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:14.962116 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.959154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:35695 (local address 127.26.189.194:36603)
0807 22:43:14.959273 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:14.959287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:14.959370 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:14.959693 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:14.959695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:14.959703 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:43:14.959786 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.959790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.960439 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.960441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.961009 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:14.961016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.961494 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.961497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:14.961562 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.961603 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.961849 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.961856 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 22:43:14.961923 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:43:14.961997 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:43:14.962119 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:14.959371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:36603 (local address 127.26.189.195:35695)
0807 22:43:14.959482 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:14.959487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:14.959513 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:43:14.959555 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:14.959557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:14.959601 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:14.959667 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:14.959857 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.960363 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.961103 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:14.961378 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:14.961471 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:14.961711 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:14.961781 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:14.961837 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:14.961952 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 22:43:14.962004 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:43:14.984586 6901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.984758 6901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:14.986956 6901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5727fda8f7344b00a1ffe22d7d0a0ca3: Sending DeleteTablet for 3 replicas of tablet e800785827d541d7960fe2769ee2589e
I20260807 22:43:14.987597 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet e800785827d541d7960fe2769ee2589e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51012
I20260807 22:43:14.987993 7736 tablet_replica.cc:330] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: stopping tablet replica
I20260807 22:43:14.988116 7736 raft_consensus.cc:2229] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.988217 7736 raft_consensus.cc:2258] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.988566 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 7012
I20260807 22:43:14.988976 7736 ts_tablet_manager.cc:1905] T e800785827d541d7960fe2769ee2589e P 4f9703e6c74d4d49b6ac3e861fe54c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.995889 7345 tablet_service.cc:1514] Processing DeleteTablet for tablet e800785827d541d7960fe2769ee2589e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60966
I20260807 22:43:14.996086 7731 tablet_replica.cc:330] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: stopping tablet replica
I20260807 22:43:14.996205 7731 raft_consensus.cc:2229] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:14.996318 7731 raft_consensus.cc:2258] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:14.997032 7731 ts_tablet_manager.cc:1905] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:14.999581 7731 ts_tablet_manager.cc:1918] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:14.999658 7731 log.cc:1196] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/e800785827d541d7960fe2769ee2589e
I20260807 22:43:14.999897 7731 ts_tablet_manager.cc:1939] T e800785827d541d7960fe2769ee2589e P fd306189caa2489484baaffe9380186b: Deleting consensus metadata
I20260807 22:43:15.000576 6879 catalog_manager.cc:4841] TS fd306189caa2489484baaffe9380186b (127.26.189.194:44751): tablet e800785827d541d7960fe2769ee2589e (table test-table [id=c567e3355c6840b9b902838616e91fd6]) successfully deleted
I20260807 22:43:15.001683 7557 tablet_service.cc:1514] Processing DeleteTablet for tablet e800785827d541d7960fe2769ee2589e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:14 UTC) from {username='oryx', principal='oryx/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53294
I20260807 22:43:15.001863 7735 tablet_replica.cc:330] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: stopping tablet replica
I20260807 22:43:15.001957 7735 raft_consensus.cc:2229] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:15.002236 7735 raft_consensus.cc:2258] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:15.003036 7735 ts_tablet_manager.cc:1905] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:15.005352 7735 ts_tablet_manager.cc:1918] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:43:15.005434 7735 log.cc:1196] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/e800785827d541d7960fe2769ee2589e
I20260807 22:43:15.005671 7735 ts_tablet_manager.cc:1939] T e800785827d541d7960fe2769ee2589e P 3fe5a1e07ca643cd9540fa220338bdd2: Deleting consensus metadata
I20260807 22:43:15.006345 6887 catalog_manager.cc:4841] TS 3fe5a1e07ca643cd9540fa220338bdd2 (127.26.189.195:35695): tablet e800785827d541d7960fe2769ee2589e (table test-table [id=c567e3355c6840b9b902838616e91fd6]) successfully deleted
W20260807 22:43:15.010337 6887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:45533 (error 108)
I20260807 22:43:15.010576 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 7238
I20260807 22:43:15.014963 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:15.013478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:45533 (local address 127.0.0.1:51030)
0807 22:43:15.014780 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:15.014862 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:45533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1216,"thread_start_us":60,"threads_started":1}
W20260807 22:43:15.018683 6887 catalog_manager.cc:4584] TS 4f9703e6c74d4d49b6ac3e861fe54c40 (127.26.189.193:45533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e800785827d541d7960fe2769ee2589e: Network error: Client connection negotiation failed: client connection to 127.26.189.193:45533: connect: Connection refused (error 111)
I20260807 22:43:15.022413 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 7469
I20260807 22:43:15.035356 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 6852
2026-08-07T22:43:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3151 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:15 dist-test-slave-4tdj krb5kdc[7852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:15 dist-test-slave-4tdj krb5kdc[7852](info): set up 2 sockets
Aug 07 22:43:15 dist-test-slave-4tdj krb5kdc[7852](info): commencing operation
krb5kdc: starting...
W20260807 22:43:16.169986 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:16 dist-test-slave-4tdj krb5kdc[7852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:43:16.286952 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43779
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:16.394982 7877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:16.395234 7877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:16.395290 7877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:16.399205 7877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:16.399261 7877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:16.399286 7877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:16.399307 7877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:16.399333 7877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:16.404493 7877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:16.405537 7877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:16.406666 7877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:16.414937 7894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:16.414942 7891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:16.415016 7890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:16.415295 7877 server_base.cc:1034] running on GCE node
I20260807 22:43:16.415863 7877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:16.416879 7877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:16.418038 7877 hybrid_clock.cc:648] HybridClock initialized: now 1786142596418024 us; error 24 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T22:43:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1394 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:16 dist-test-slave-4tdj krb5kdc[7900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:16 dist-test-slave-4tdj krb5kdc[7900](info): set up 2 sockets
Aug 07 22:43:16 dist-test-slave-4tdj krb5kdc[7900](info): commencing operation
krb5kdc: starting...
W20260807 22:43:17.559309 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:17 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:43:17.674582 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36329
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:17.781996 7923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:17.782243 7923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:17.782317 7923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:17.786226 7923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:17.786279 7923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:17.786322 7923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:17.786346 7923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:17.786374 7923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:17.791865 7923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:17.792867 7923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:17.793833 7923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:17.801792 7932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:17.801781 7931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:17.801882 7923 server_base.cc:1034] running on GCE node
W20260807 22:43:17.802002 7935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:17.802605 7923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:17.803618 7923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:17.804771 7923 hybrid_clock.cc:648] HybridClock initialized: now 1786142597804758 us; error 26 us; skew 500 ppm
Aug 07 22:43:17 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:17.807363 7923 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:43:17.808132 7923 webserver.cc:460] Webserver started at http://127.26.189.254:40293/ using document root <none> and password file <none>
I20260807 22:43:17.808629 7923 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:17.808686 7923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:17.808848 7923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:17.810751 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "36a48c58cce34f64afc1127796451dac"
format_stamp: "Formatted at 2026-08-07 22:43:17 on dist-test-slave-4tdj"
server_key: "cd2f0af6934e0f956330f471842357bd"
server_key_iv: "1c4d4bd8f5b56469462221fb1e2f782e"
server_key_version: "encryptionkey@0"
I20260807 22:43:17.811247 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "36a48c58cce34f64afc1127796451dac"
format_stamp: "Formatted at 2026-08-07 22:43:17 on dist-test-slave-4tdj"
server_key: "cd2f0af6934e0f956330f471842357bd"
server_key_iv: "1c4d4bd8f5b56469462221fb1e2f782e"
server_key_version: "encryptionkey@0"
I20260807 22:43:17.814610 7923 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:17.816805 7945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:17.817935 7923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:17.818038 7923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "36a48c58cce34f64afc1127796451dac"
format_stamp: "Formatted at 2026-08-07 22:43:17 on dist-test-slave-4tdj"
server_key: "cd2f0af6934e0f956330f471842357bd"
server_key_iv: "1c4d4bd8f5b56469462221fb1e2f782e"
server_key_version: "encryptionkey@0"
I20260807 22:43:17.818171 7923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:17.831879 7923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:17.834760 7923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:17.834921 7923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:17.841416 7923 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:36329
I20260807 22:43:17.841430 8044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:36329 every 8 connection(s)
I20260807 22:43:17.842278 7923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:17.844937 8046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:17.850160 8046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Bootstrap starting.
I20260807 22:43:17.850528 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 7923
I20260807 22:43:17.850638 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:17.850920 27383 external_mini_cluster.cc:1428] Setting key e70520dcb96425bf491ade5bae097d97
I20260807 22:43:17.852562 8046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:17.853432 8046 log.cc:824] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:17.855463 8046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: No bootstrap required, opened a new log
I20260807 22:43:17.859202 8046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:17.859408 8046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:17.859443 8046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a48c58cce34f64afc1127796451dac, State: Initialized, Role: FOLLOWER
Aug 07 22:43:17 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:17.859871 8046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:17.860034 8046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:17.860102 8046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:17.860215 8046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:17.860993 8046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:17.861337 8046 leader_election.cc:302] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a48c58cce34f64afc1127796451dac; no voters:
I20260807 22:43:17.861657 8046 leader_election.cc:288] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:17.861739 8053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:17.861944 8053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 LEADER]: Becoming Leader. State: Replica: 36a48c58cce34f64afc1127796451dac, State: Running, Role: LEADER
I20260807 22:43:17.862313 8053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:17.862905 8046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:17.863897 8054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a48c58cce34f64afc1127796451dac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } } }
I20260807 22:43:17.864019 8054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:17.864521 8057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a48c58cce34f64afc1127796451dac. Latest consensus state: current_term: 1 leader_uuid: "36a48c58cce34f64afc1127796451dac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } } }
I20260807 22:43:17.865113 8057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:17.865968 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:17.852174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.26.189.254:36329)
0807 22:43:17.852615 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:17.852636 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:17.852678 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:43:17.853455 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:17.853478 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:17.853877 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:17.854171 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:17.854855 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.856260 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.857079 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:17.857780 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:17.857937 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:17.860295 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:17.860302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:17.860318 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:17.860347 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:17.862178 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.862694 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.862700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.862705 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.862844 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:17.863218 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:17.863221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:17.863223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:17.863752 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:17.864513 (+ 761us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:17.864691 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:43:17.864977 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
W20260807 22:43:17.867666 8082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:43:17.867739 8082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:43:17.867813 8070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:17.868584 8070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:17.873507 8070 catalog_manager.cc:1347] Generated new cluster ID: ddbde60fa9b74920ac99c26b6370a378
I20260807 22:43:17.873565 8070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:17.882807 8070 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:17.883617 8070 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:17.888562 8070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Generated new TSK 0
I20260807 22:43:17.889151 8070 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:43:17.927129 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36329
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:18.063752 8088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:18.063995 8088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:18.064049 8088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:18.067972 8088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:18.068024 8088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:18.068090 8088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:18.073117 8088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:18.074043 8088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:18.075008 8088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:18.083817 8095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:18.083909 8088 server_base.cc:1034] running on GCE node
W20260807 22:43:18.083825 8099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:18.083825 8096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:18.084407 8088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:18.085011 8088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:18.086158 8088 hybrid_clock.cc:648] HybridClock initialized: now 1786142598086145 us; error 27 us; skew 500 ppm
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:18.088711 8088 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:43:18.089483 8088 webserver.cc:460] Webserver started at http://127.26.189.193:45777/ using document root <none> and password file <none>
I20260807 22:43:18.089972 8088 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:18.090034 8088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:18.090220 8088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:18.092203 8088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "b4851adab2524f118fd17dd6c192baf0"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "44020aa56dc04c320fcbb806958139de"
server_key_iv: "cc2673561d113aacb9c9dfe4b6b85037"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.092742 8088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "b4851adab2524f118fd17dd6c192baf0"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "44020aa56dc04c320fcbb806958139de"
server_key_iv: "cc2673561d113aacb9c9dfe4b6b85037"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.096304 8088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:18.098512 8111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.099622 8088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:18.099726 8088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "b4851adab2524f118fd17dd6c192baf0"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "44020aa56dc04c320fcbb806958139de"
server_key_iv: "cc2673561d113aacb9c9dfe4b6b85037"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.099815 8088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:18.116514 8088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:18.119536 8088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:18.119702 8088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:18.120245 8088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:18.121029 8088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:18.121078 8088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.121145 8088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:18.121168 8088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.129509 8088 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:39947
I20260807 22:43:18.129534 8281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:39947 every 8 connection(s)
I20260807 22:43:18.130332 8088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:18.133862 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 8088
I20260807 22:43:18.133998 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:18.134361 27383 external_mini_cluster.cc:1428] Setting key 6e28208f47ea661825e1922cbfab13f4
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:18.143946 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:18.131955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:45197 (local address 127.26.189.254:36329)
0807 22:43:18.132095 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:18.132100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:18.132900 (+ 800us) server_negotiation.cc:407] Connection header received
0807 22:43:18.134394 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:18.134397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:18.134456 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:18.134528 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:18.135928 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.136627 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.137828 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.138056 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.138135 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:18.140635 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:18.140638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:18.140640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:18.140662 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:18.141981 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:18.142506 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:18.142509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:18.142510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:18.142595 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:18.142911 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:18.142913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:18.142914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:18.143086 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:18.143173 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:18.143656 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 22:43:18.143785 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:43:18.144759 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:18.132334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:45197)
0807 22:43:18.132754 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:18.132795 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:18.134118 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:18.134730 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:18.134740 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:18.135186 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:18.135780 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:18.135795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.136758 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.136761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:18.137709 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:18.137720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.138192 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.138196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:18.138403 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:18.138958 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:18.138973 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:18.140466 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:18.142099 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:18.142105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:18.142119 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:18.142410 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:18.142676 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:18.142680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:18.142682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:18.142815 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:18.143186 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:18.143192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:18.143420 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 22:43:18.143571 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 22:43:18.143871 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
I20260807 22:43:18.145831 8282 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36329
I20260807 22:43:18.146046 8282 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:18.146543 8282 heartbeater.cc:511] Master 127.26.189.254:36329 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:43:18.148011 7978 ts_manager.cc:194] Registered new tserver with Master: b4851adab2524f118fd17dd6c192baf0 (127.26.189.193:39947)
I20260807 22:43:18.149080 7978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:45197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:43:18.161196 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36329
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:18.269409 8292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:18.269644 8292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:18.269686 8292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:18.273547 8292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:18.273598 8292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:18.273672 8292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:18.278678 8292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:18.279596 8292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:18.280681 8292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:18.290575 8309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:18.290604 8303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:18.290619 8304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:18.291317 8292 server_base.cc:1034] running on GCE node
I20260807 22:43:18.291720 8292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:18.292323 8292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:18.293481 8292 hybrid_clock.cc:648] HybridClock initialized: now 1786142598293465 us; error 26 us; skew 500 ppm
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:18.296073 8292 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:43:18.296876 8292 webserver.cc:460] Webserver started at http://127.26.189.194:42769/ using document root <none> and password file <none>
I20260807 22:43:18.297389 8292 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:18.297452 8292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:18.297654 8292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:18.299633 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "125782b3500c44b995eccae995b22bfd"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "9be1c4be069ccf5157b94ae1da703f1b"
server_key_iv: "99b95be641cabcac72da9c811b966048"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.300158 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "125782b3500c44b995eccae995b22bfd"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "9be1c4be069ccf5157b94ae1da703f1b"
server_key_iv: "99b95be641cabcac72da9c811b966048"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.303665 8292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:43:18.305842 8318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.306947 8292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:43:18.307049 8292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "125782b3500c44b995eccae995b22bfd"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "9be1c4be069ccf5157b94ae1da703f1b"
server_key_iv: "99b95be641cabcac72da9c811b966048"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.307137 8292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:18.348399 8292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:18.351611 8292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:18.351794 8292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:18.352408 8292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:18.353240 8292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:18.353289 8292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.353343 8292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:18.353369 8292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.361732 8292 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:45253
I20260807 22:43:18.361760 8485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:45253 every 8 connection(s)
I20260807 22:43:18.362584 8292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:18.367949 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 8292
I20260807 22:43:18.368046 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:18.368352 27383 external_mini_cluster.cc:1428] Setting key b1cbee942cb6e57b7d9360cbf05a1531
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:18.376189 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:18.364383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:51975 (local address 127.26.189.254:36329)
0807 22:43:18.364532 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:18.364537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:18.365424 (+ 887us) server_negotiation.cc:407] Connection header received
0807 22:43:18.366808 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:18.366812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:18.366863 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:18.366946 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:18.368076 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.368796 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.370006 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.370287 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.370408 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:18.373065 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:18.373069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:18.373070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:18.373095 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:18.374313 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:18.374780 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:18.374784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:18.374786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:18.374865 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:18.375167 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:18.375170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:18.375172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:18.375347 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:18.375432 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:18.375901 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 22:43:18.376023 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:43:18.376921 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:18.364800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:51975)
0807 22:43:18.365261 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:18.365304 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:18.366577 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:18.367130 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:18.367136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:18.367483 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:18.367933 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:18.367946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.368923 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.368926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:18.369887 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:18.369897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.370432 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.370435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:18.370578 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:18.371101 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:18.371118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:18.372911 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:18.374415 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:18.374421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:18.374435 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:18.374687 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:18.374945 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:18.374948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:18.374950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:18.375080 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:18.375443 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:18.375449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:18.375687 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 22:43:18.375844 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:43:18.376137 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:43:18.378078 8486 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36329
I20260807 22:43:18.378331 8486 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:18.378794 8486 heartbeater.cc:511] Master 127.26.189.254:36329 requested a full tablet report, sending...
I20260807 22:43:18.379793 7978 ts_manager.cc:194] Registered new tserver with Master: 125782b3500c44b995eccae995b22bfd (127.26.189.194:45253)
I20260807 22:43:18.380386 7978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:51975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:43:18.395049 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36329
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:18.503548 8493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:18.503788 8493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:18.503832 8493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:18.507794 8493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:18.507850 8493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:18.507916 8493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:18.513060 8493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:18.514029 8493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:18.515021 8493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:18.524091 8506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:18.524081 8504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:18.524083 8502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:18.524248 8493 server_base.cc:1034] running on GCE node
I20260807 22:43:18.524741 8493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:18.525362 8493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:18.526507 8493 hybrid_clock.cc:648] HybridClock initialized: now 1786142598526483 us; error 39 us; skew 500 ppm
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:18.529085 8493 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:43:18.529908 8493 webserver.cc:460] Webserver started at http://127.26.189.195:40011/ using document root <none> and password file <none>
I20260807 22:43:18.530454 8493 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:18.530521 8493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:18.530720 8493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:18.532658 8493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "fd9de9ddc46540ad932e45eaf9059144"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "f2893ebfcb4051ef577380f7a54e31c4"
server_key_iv: "e299ec08f21884ca86d86fc010a966b8"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.533175 8493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "fd9de9ddc46540ad932e45eaf9059144"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "f2893ebfcb4051ef577380f7a54e31c4"
server_key_iv: "e299ec08f21884ca86d86fc010a966b8"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.536701 8493 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 22:43:18.539184 8519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.540313 8493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260807 22:43:18.540417 8493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "fd9de9ddc46540ad932e45eaf9059144"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "f2893ebfcb4051ef577380f7a54e31c4"
server_key_iv: "e299ec08f21884ca86d86fc010a966b8"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.540509 8493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:18.552292 8493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:18.555116 8493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:18.555285 8493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:18.555816 8493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:18.556602 8493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:18.556649 8493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.556699 8493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:18.556727 8493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.565274 8493 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:43985
I20260807 22:43:18.565292 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:43985 every 8 connection(s)
I20260807 22:43:18.566102 8493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:18.571189 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 8493
I20260807 22:43:18.571334 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:18.571631 27383 external_mini_cluster.cc:1428] Setting key d8a31495e16a7bc57d59aadd8f641bee
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:18.578775 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:18.567787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:48031 (local address 127.26.189.254:36329)
0807 22:43:18.567915 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:18.567919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:18.568697 (+ 778us) server_negotiation.cc:407] Connection header received
0807 22:43:18.569918 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:18.569921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:18.569967 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:18.570036 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:18.571159 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.571912 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.573364 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.573645 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.573746 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:18.575917 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:18.575921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:18.575923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:18.575946 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:18.577166 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:18.577570 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:18.577572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:18.577574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:18.577656 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:18.577895 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:18.577897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:18.577898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:18.578051 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:18.578122 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:18.578558 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 22:43:18.578661 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:43:18.579504 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:18.568129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:48031)
0807 22:43:18.568545 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:18.568587 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:18.569743 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:18.570179 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:18.570185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:18.570540 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:18.571026 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:18.571039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.572060 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.572064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:18.573257 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:18.573268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:18.573782 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:18.573785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:18.573918 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:18.574445 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:18.574460 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:18.575781 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:18.577258 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:18.577263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:18.577273 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:18.577501 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:18.577716 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:18.577718 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:18.577720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:18.577835 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:18.578122 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:18.578126 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:18.578371 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 22:43:18.578521 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:43:18.578747 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":106,"threads_started":1}
I20260807 22:43:18.580515 8685 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36329
I20260807 22:43:18.580717 8685 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:18.581130 8685 heartbeater.cc:511] Master 127.26.189.254:36329 requested a full tablet report, sending...
I20260807 22:43:18.582031 7978 ts_manager.cc:194] Registered new tserver with Master: fd9de9ddc46540ad932e45eaf9059144 (127.26.189.195:43985)
I20260807 22:43:18.582574 7978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:48031
I20260807 22:43:18.585078 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab.
Entry for principal oryx/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab.
WARNING: no policy specified for oryx/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab.
Entry for principal oryx/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab.
WARNING: no policy specified for oryx/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab.
Entry for principal oryx/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab.
I20260807 22:43:18.650544 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 8088
I20260807 22:43:18.666515 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 8292
I20260807 22:43:18.675607 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 8493
I20260807 22:43:18.685035 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 7923
I20260807 22:43:18.694730 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36329
--webserver_interface=127.26.189.254
--webserver_port=40293
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:18.802474 8704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:18.802717 8704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:18.802769 8704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:18.806623 8704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:18.806668 8704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:18.806694 8704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:18.806713 8704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:18.806733 8704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:18.811725 8704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=40293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:18.812629 8704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:18.813544 8704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:18.821693 8711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:18.821743 8704 server_base.cc:1034] running on GCE node
W20260807 22:43:18.821928 8712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:18.821700 8716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:18.822672 8704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:18.823698 8704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:18.824853 8704 hybrid_clock.cc:648] HybridClock initialized: now 1786142598824837 us; error 31 us; skew 500 ppm
Aug 07 22:43:18 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142598, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:18.827517 8704 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:43:18.828357 8704 webserver.cc:460] Webserver started at http://127.26.189.254:40293/ using document root <none> and password file <none>
I20260807 22:43:18.828860 8704 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:18.828943 8704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:18.833324 8704 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:43:18.835341 8723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:18.836493 8704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:18.836603 8704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "36a48c58cce34f64afc1127796451dac"
format_stamp: "Formatted at 2026-08-07 22:43:17 on dist-test-slave-4tdj"
server_key: "cd2f0af6934e0f956330f471842357bd"
server_key_iv: "1c4d4bd8f5b56469462221fb1e2f782e"
server_key_version: "encryptionkey@0"
I20260807 22:43:18.837100 8704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:18.863597 8704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:18.866351 8704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:18.866533 8704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:18.873416 8704 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:36329
I20260807 22:43:18.873432 8794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:36329 every 8 connection(s)
I20260807 22:43:18.874483 8704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:18.878608 8796 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:43:18.880169 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Bootstrap starting.
I20260807 22:43:18.880467 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 8704
I20260807 22:43:18.880865 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:39947
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:36329
--webserver_port=45777
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:18.896911 8796 log.cc:824] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:18.903394 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:43:18.903721 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Bootstrap complete.
I20260807 22:43:18.907841 8796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:18.908224 8796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a48c58cce34f64afc1127796451dac, State: Initialized, Role: FOLLOWER
I20260807 22:43:18.908699 8796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:18.908821 8796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:18.908887 8796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:18.908981 8796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:43:18.910869 8796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:18.911166 8796 leader_election.cc:302] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a48c58cce34f64afc1127796451dac; no voters:
I20260807 22:43:18.911442 8796 leader_election.cc:288] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:43:18.911584 8802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:43:18.911928 8802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [term 2 LEADER]: Becoming Leader. State: Replica: 36a48c58cce34f64afc1127796451dac, State: Running, Role: LEADER
I20260807 22:43:18.912290 8802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } }
I20260807 22:43:18.912575 8796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:18.913630 8804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a48c58cce34f64afc1127796451dac. Latest consensus state: current_term: 2 leader_uuid: "36a48c58cce34f64afc1127796451dac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } } }
I20260807 22:43:18.913750 8804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:18.914433 8803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "36a48c58cce34f64afc1127796451dac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a48c58cce34f64afc1127796451dac" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36329 } } }
I20260807 22:43:18.914513 8803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:18.916913 8831 catalog_manager.cc:1259] Loaded cluster ID: ddbde60fa9b74920ac99c26b6370a378
I20260807 22:43:18.916961 8831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: loading cluster ID for follower catalog manager: success
I20260807 22:43:18.918669 8831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: acquiring CA information for follower catalog manager: success
I20260807 22:43:18.919690 8831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:43:18.919835 8820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:18.920622 8820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:18.920993 8820 catalog_manager.cc:1259] Loaded cluster ID: ddbde60fa9b74920ac99c26b6370a378
I20260807 22:43:18.921048 8820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:18.921562 8820 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:18.921936 8820 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a48c58cce34f64afc1127796451dac: Loaded TSK: 0
I20260807 22:43:18.922580 8820 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:43:19.004002 8798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:19.004241 8798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:19.004300 8798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:19.008250 8798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:19.008303 8798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:19.008380 8798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:19.013510 8798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.193.keytab
--principal=oryx/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:39947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=45777
--tserver_master_addrs=127.26.189.254:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:19.014487 8798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:19.015429 8798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:19.026049 8843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:19.026108 8841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:19.026396 8839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:19.031359 8798 server_base.cc:1034] running on GCE node
I20260807 22:43:19.031842 8798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:19.032620 8798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:19.033824 8798 hybrid_clock.cc:648] HybridClock initialized: now 1786142599033804 us; error 32 us; skew 500 ppm
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142599, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:19.037279 8798 init.cc:377] Logged in from keytab as oryx/127.26.189.193@KRBTEST.COM (short username oryx)
I20260807 22:43:19.038422 8798 webserver.cc:460] Webserver started at http://127.26.189.193:45777/ using document root <none> and password file <none>
I20260807 22:43:19.039129 8798 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:19.039242 8798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:19.044296 8798 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:19.046825 8857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.048249 8798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:19.048360 8798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "b4851adab2524f118fd17dd6c192baf0"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "44020aa56dc04c320fcbb806958139de"
server_key_iv: "cc2673561d113aacb9c9dfe4b6b85037"
server_key_version: "encryptionkey@0"
I20260807 22:43:19.049014 8798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:19.074589 8798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:19.077786 8798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:19.077937 8798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:19.078500 8798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:19.079283 8798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:19.079329 8798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.079367 8798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:19.079387 8798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.087642 8798 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:39947
I20260807 22:43:19.087680 9037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:39947 every 8 connection(s)
I20260807 22:43:19.088557 8798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.098152 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 8798
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.098579 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:45253
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:36329
--webserver_port=42769
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:19.100275 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.090525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:45755)
0807 22:43:19.090984 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.091033 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.092234 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.093176 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.093182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.093492 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.093911 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.093924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.095087 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.095090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.095910 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.095919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.096524 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.096527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.096649 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.097085 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.097099 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.099477 (+ 2378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":112,"threads_started":1}
W20260807 22:43:19.100538 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.090434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:45755 (local address 127.26.189.254:36329)
0807 22:43:19.090842 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.090858 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.091130 (+ 272us) server_negotiation.cc:407] Connection header received
0807 22:43:19.092554 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.092565 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.092845 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.093096 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.094048 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.094986 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.096000 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.096417 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.096521 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.099731 (+ 3210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:45755: BlockingRecv error: recv got EOF from 127.26.189.193:45755 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":100,"threads_started":1}
W20260807 22:43:19.101065 9039 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:36329 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.189.254:36329: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.106144 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.101461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:59905)
0807 22:43:19.101669 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.101680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.101746 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.101940 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.101942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.102056 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.102152 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.102157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.102857 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.102861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.103391 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.103399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.103856 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.103860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.103946 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.104360 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.104370 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.106036 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260807 22:43:19.106279 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.101565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:59905 (local address 127.26.189.254:36329)
0807 22:43:19.101682 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.101686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.101719 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:43:19.101806 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.101809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.101853 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.101931 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.102238 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.102755 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.103497 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.103749 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.103832 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.106183 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:59905: BlockingRecv error: recv got EOF from 127.26.189.193:59905 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.111096 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.106784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:51517)
0807 22:43:19.106909 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.106922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.106989 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.107199 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.107202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.107327 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.107387 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.107391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.108040 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.108043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.108526 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.108532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.108925 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.108929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.109000 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.109393 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.109403 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.110995 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260807 22:43:19.111227 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.106878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:51517 (local address 127.26.189.254:36329)
0807 22:43:19.106985 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.106989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.107001 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:19.107066 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.107069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.107115 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.107175 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.107483 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.107945 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.108622 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.108836 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.108915 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.111132 (+ 2217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:51517: BlockingRecv error: recv got EOF from 127.26.189.193:51517 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 22:43:19.111274 9039 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:43:19.207269 9045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:19.207494 9045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:19.207543 9045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:19.211462 9045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:19.211510 9045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:19.211588 9045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:19.216691 9045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.194.keytab
--principal=oryx/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:45253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=42769
--tserver_master_addrs=127.26.189.254:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:19.217613 9045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:19.218603 9045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:19.227885 9055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:19.227887 9054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:19.228107 9045 server_base.cc:1034] running on GCE node
W20260807 22:43:19.227929 9057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:19.228595 9045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:19.229187 9045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:19.230346 9045 hybrid_clock.cc:648] HybridClock initialized: now 1786142599230322 us; error 36 us; skew 500 ppm
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142599, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:19.232863 9045 init.cc:377] Logged in from keytab as oryx/127.26.189.194@KRBTEST.COM (short username oryx)
I20260807 22:43:19.233702 9045 webserver.cc:460] Webserver started at http://127.26.189.194:42769/ using document root <none> and password file <none>
I20260807 22:43:19.234196 9045 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:19.234280 9045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:19.237880 9045 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 22:43:19.239817 9066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.240970 9045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:19.241075 9045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "125782b3500c44b995eccae995b22bfd"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "9be1c4be069ccf5157b94ae1da703f1b"
server_key_iv: "99b95be641cabcac72da9c811b966048"
server_key_version: "encryptionkey@0"
I20260807 22:43:19.241596 9045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:19.273056 9045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:19.275635 9045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:19.275804 9045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:19.276365 9045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:19.277155 9045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:19.277204 9045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.277256 9045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:19.277285 9045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.285584 9045 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:45253
I20260807 22:43:19.285609 9232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:45253 every 8 connection(s)
I20260807 22:43:19.286537 9045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.296070 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 9045
W20260807 22:43:19.296530 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.288194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:33785 (local address 127.26.189.254:36329)
0807 22:43:19.288315 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.288319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.289111 (+ 792us) server_negotiation.cc:407] Connection header received
0807 22:43:19.290359 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.290362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.290414 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.290485 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.291540 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.292062 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.293078 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.293281 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.293348 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.296401 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:33785: BlockingRecv error: recv got EOF from 127.26.189.194:33785 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:43:19.296561 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:43985
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:36329
--webserver_port=40011
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:43:19.297026 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.288520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:33785)
0807 22:43:19.288955 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.288996 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.290153 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.290631 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.290637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.290961 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.291425 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.291438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.292153 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.292156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.292991 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.292999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.293384 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.293386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.293517 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.294030 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.294046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.296230 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
W20260807 22:43:19.297562 9233 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:36329 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.189.254:36329: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.302529 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.297962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:49849)
0807 22:43:19.298052 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.298066 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.298145 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.298378 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.298381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.298530 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.298620 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.298625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.299299 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.299303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.299833 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.299841 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.300235 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.300238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.300325 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.300735 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.300749 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.302399 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 22:43:19.302595 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.298066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:49849 (local address 127.26.189.254:36329)
0807 22:43:19.298186 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.298190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.298202 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:19.298240 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.298242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.298276 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.298366 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.298726 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.299206 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.299937 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.300143 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.300225 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.302507 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:49849: BlockingRecv error: recv got EOF from 127.26.189.194:49849 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.307241 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.303064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:38195)
0807 22:43:19.303209 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.303221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.303287 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.303478 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.303480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.303610 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.303675 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.303679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.304327 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.304330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.304815 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.304822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.305199 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.305215 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.305288 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.305657 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.305668 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.307135 (+ 1467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260807 22:43:19.307343 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.303170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:38195 (local address 127.26.189.254:36329)
0807 22:43:19.303281 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.303285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.303298 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:19.303354 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.303357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.303394 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.303454 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.303772 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.304233 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.304913 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.305109 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.305188 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.307247 (+ 2059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:38195: BlockingRecv error: recv got EOF from 127.26.189.194:38195 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 22:43:19.307610 9233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:43:19.407212 9237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:19.407470 9237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:19.407514 9237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:19.411425 9237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:19.411489 9237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:19.411556 9237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:19.416863 9237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/krb5kdc/oryx_127.26.189.195.keytab
--principal=oryx/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:43985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=40011
--tserver_master_addrs=127.26.189.254:36329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:19.417811 9237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:19.418823 9237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:19.428213 9245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:19.428287 9237 server_base.cc:1034] running on GCE node
W20260807 22:43:19.428213 9251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:19.428246 9247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:19.428869 9237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:19.429495 9237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:19.430649 9237 hybrid_clock.cc:648] HybridClock initialized: now 1786142599430622 us; error 40 us; skew 500 ppm
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142599, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:19.433323 9237 init.cc:377] Logged in from keytab as oryx/127.26.189.195@KRBTEST.COM (short username oryx)
I20260807 22:43:19.434252 9237 webserver.cc:460] Webserver started at http://127.26.189.195:40011/ using document root <none> and password file <none>
I20260807 22:43:19.434836 9237 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:19.434926 9237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:19.438792 9237 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 22:43:19.440806 9262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.442020 9237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:19.442126 9237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "fd9de9ddc46540ad932e45eaf9059144"
format_stamp: "Formatted at 2026-08-07 22:43:18 on dist-test-slave-4tdj"
server_key: "f2893ebfcb4051ef577380f7a54e31c4"
server_key_iv: "e299ec08f21884ca86d86fc010a966b8"
server_key_version: "encryptionkey@0"
I20260807 22:43:19.442688 9237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:19.455768 9237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:19.458281 9237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:19.458479 9237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:19.459028 9237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:19.459831 9237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:19.459879 9237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.459929 9237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:19.459959 9237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:19.468601 9237 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:43985
I20260807 22:43:19.468639 9445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:43985 every 8 connection(s)
I20260807 22:43:19.469539 9237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:19.473158 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 9237
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:43:19.480243 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.471268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:44903 (local address 127.26.189.254:36329)
0807 22:43:19.471423 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.471427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.472209 (+ 782us) server_negotiation.cc:407] Connection header received
0807 22:43:19.473484 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.473487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.473541 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.473611 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.475199 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.475720 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.476792 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.476996 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.477076 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.480124 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:44903: BlockingRecv error: recv got EOF from 127.26.189.195:44903 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:43:19.480798 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.471631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:44903)
0807 22:43:19.472048 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.472089 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.473271 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.473809 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.473817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.474581 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.475051 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.475064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.475835 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.475839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.476675 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.476685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.477134 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.477140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.477285 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.477717 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.477732 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.480004 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
W20260807 22:43:19.481218 9449 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:36329 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.189.254:36329: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
I20260807 22:43:19.482331 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.474088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.26.189.254:36329)
0807 22:43:19.474346 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.474351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.474364 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:19.474403 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.474405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.474452 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.474540 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.475236 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.475754 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.476607 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.476827 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.476910 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.478282 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:19.478287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:19.478322 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:19.478352 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:19.480008 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:19.480501 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:19.480506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:19.480510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:19.480641 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:19.480923 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:19.480927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:19.480928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:19.481314 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:19.481501 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:19.481897 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 22:43:19.482187 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.486573 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.481603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:43159)
0807 22:43:19.481742 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.481754 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.481822 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.482011 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.482014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.482180 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.482262 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.482267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.483181 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.483184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.483978 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.483988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.484454 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.484457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.484525 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.484896 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.484910 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.486466 (+ 1556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260807 22:43:19.486686 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.481686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:43159 (local address 127.26.189.254:36329)
0807 22:43:19.481791 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.481795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.481807 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:19.481890 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.481893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.481929 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.481987 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.482376 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.483068 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.484097 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.484349 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.484440 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.486580 (+ 2140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:43159: BlockingRecv error: recv got EOF from 127.26.189.195:43159 (error 108)
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":41}
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:19 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:19.491124 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:19.487018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:41433)
0807 22:43:19.487120 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:19.487132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:19.487205 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:19.487388 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:19.487391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:19.487523 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:19.487592 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.487598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.488240 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.488244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.488743 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:19.488750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.489133 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.489137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:19.489210 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.489600 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:19.489612 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:19.491009 (+ 1397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 22:43:19.491214 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:19.487088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:41433 (local address 127.26.189.254:36329)
0807 22:43:19.487220 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:19.487224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:19.487237 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:19.487268 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:19.487271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:19.487304 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:19.487363 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:19.487691 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.488146 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.488841 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:19.489037 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:19.489117 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:19.491124 (+ 2007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:41433: BlockingRecv error: recv got EOF from 127.26.189.195:41433 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 22:43:19.491312 9449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 22:43:20 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:20 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:20.117143 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.111848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:39971)
0807 22:43:20.112120 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:20.112134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:20.112228 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:20.112512 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:20.112515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:20.112739 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:20.112869 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.112876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.113608 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.113627 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.114241 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.114251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.114685 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.114687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.114761 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.115245 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:20.115257 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:20.117029 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":73,"threads_started":1}
W20260807 22:43:20.117343 9456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:20.111898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:39971 (local address 127.26.189.254:36329)
0807 22:43:20.112145 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:20.112148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:20.112183 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:43:20.112289 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:20.112291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:20.112332 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:20.112405 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:20.112971 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.113507 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.114377 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.114595 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.114672 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.117212 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:39971: BlockingRecv error: recv got EOF from 127.26.189.193:39971 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
Aug 07 22:43:20 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:20 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:20.313769 9459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.308190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:33613)
0807 22:43:20.308455 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:20.308470 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:20.308588 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:20.308821 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:20.308823 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:20.309028 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:20.309128 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.309134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.309893 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.309897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.310669 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.310680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.311185 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.311189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.311262 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.311723 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:20.311733 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:20.313650 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
W20260807 22:43:20.313884 9456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:20.308301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:33613 (local address 127.26.189.254:36329)
0807 22:43:20.308442 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:20.308446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:20.308543 (+ 97us) server_negotiation.cc:407] Connection header received
0807 22:43:20.308675 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:20.308678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:20.308722 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:20.308787 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:20.309238 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.309773 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.310772 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.311061 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.311165 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.313772 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:33613: BlockingRecv error: recv got EOF from 127.26.189.194:33613 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 22:43:20 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:20 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:20.497092 9460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:20.491919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:42285)
0807 22:43:20.492210 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:20.492229 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:20.492330 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:20.492571 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:20.492573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:20.492774 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:20.492871 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.492876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.493603 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.493607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.494185 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:20.494193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.494649 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.494653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:20.494734 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.495194 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:20.495205 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:20.496929 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
W20260807 22:43:20.497179 9456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:20.491996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:42285 (local address 127.26.189.254:36329)
0807 22:43:20.492129 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:20.492133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:20.492293 (+ 160us) server_negotiation.cc:407] Connection header received
0807 22:43:20.492423 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:20.492426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:20.492476 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:20.492536 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:20.492982 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.493504 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.494315 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:20.494547 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:20.494641 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:20.497076 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:42285: BlockingRecv error: recv got EOF from 127.26.189.195:42285 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 22:43:21 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:21 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:21.122936 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.117866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:58665)
0807 22:43:21.118134 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:21.118147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:21.118238 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:21.118475 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:21.118477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:21.118670 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:21.118772 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.118777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.119476 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.119478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.120038 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.120046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.120400 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.120402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.120467 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.120965 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:21.120976 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:21.122785 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":84,"threads_started":1}
W20260807 22:43:21.123028 9464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:21.117934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:58665 (local address 127.26.189.254:36329)
0807 22:43:21.118164 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.118168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.118195 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:43:21.118328 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.118330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.118375 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.118448 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.118862 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.119399 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.120115 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.120322 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.120383 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.122918 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:58665: BlockingRecv error: recv got EOF from 127.26.189.193:58665 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":61,"threads_started":1}
Aug 07 22:43:21 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:21 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:21.321120 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.314493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:55221)
0807 22:43:21.314806 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:21.314824 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:21.314932 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:21.315341 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:21.315345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:21.315574 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:21.315706 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.315716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.316682 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.316685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.317508 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.317521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.318114 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.318117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.318203 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.318704 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:21.318718 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:21.320920 (+ 2202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":71,"threads_started":1}
W20260807 22:43:21.321229 9464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:21.314562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:55221 (local address 127.26.189.254:36329)
0807 22:43:21.314867 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.314872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.314888 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:43:21.315041 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.315044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.315097 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.315187 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.315835 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.316549 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.317640 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.317985 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.318103 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.321105 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:55221: BlockingRecv error: recv got EOF from 127.26.189.194:55221 (error 108)
Metrics: {"server-negotiator.queue_time_us":183}
Aug 07 22:43:21 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:21 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:21.511652 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:21.498030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:48581)
0807 22:43:21.498378 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:21.498391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:21.498497 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:21.498746 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:21.498749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:21.498945 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:21.499043 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.499049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.499786 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.499790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.500339 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:21.500346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.500768 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.500772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:21.500852 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.509762 (+ 8910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:21.509774 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:21.511505 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":120,"threads_started":1}
W20260807 22:43:21.511744 9464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:21.498145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:48581 (local address 127.26.189.254:36329)
0807 22:43:21.498270 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:21.498275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:21.498458 (+ 183us) server_negotiation.cc:407] Connection header received
0807 22:43:21.498584 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:21.498587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:21.498644 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:21.498719 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:21.499166 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.499683 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.500447 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:21.500669 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:21.500755 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:21.511648 (+ 10893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:48581: BlockingRecv error: recv got EOF from 127.26.189.195:48581 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 22:43:22 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:22 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:22.128765 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:22.123603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:52643)
0807 22:43:22.123872 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:22.123885 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:22.123973 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:22.124182 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:22.124184 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:22.124358 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:22.124454 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:22.124459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.125184 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.125188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:22.125791 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:22.125801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.126199 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.126203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:22.126322 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:22.126846 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:22.126861 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:22.128641 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
W20260807 22:43:22.128901 9469 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:22.123695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:52643 (local address 127.26.189.254:36329)
0807 22:43:22.123913 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:22.123916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:22.123929 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:22.124045 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:22.124047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:22.124088 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:22.124160 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:22.124536 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.125068 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.125886 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.126104 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.126174 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:22.128772 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:52643: BlockingRecv error: recv got EOF from 127.26.189.193:52643 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":57,"threads_started":1}
Aug 07 22:43:22 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:22 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:22.329555 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:22.324379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:52029)
0807 22:43:22.324659 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:22.324677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:22.324802 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:22.325072 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:22.325075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:22.325303 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:22.325413 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:22.325420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.326245 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.326248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:22.326874 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:22.326884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.327267 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.327269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:22.327331 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:22.327781 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:22.327792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:22.329435 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
W20260807 22:43:22.329684 9469 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:22.324489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:52029 (local address 127.26.189.254:36329)
0807 22:43:22.324662 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:22.324668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:22.324747 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:43:22.324889 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:22.324893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:22.324947 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:22.325038 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:22.325592 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.326139 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.326970 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.327184 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.327256 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:22.329568 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:52029: BlockingRecv error: recv got EOF from 127.26.189.194:52029 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 22:43:22 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:22 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:22.517930 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:22.512448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:41255)
0807 22:43:22.512702 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:22.512715 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:22.512813 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:22.513016 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:22.513018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:22.513196 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:22.513297 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:22.513302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.514122 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.514126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:22.514946 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:22.514959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.515522 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.515526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:22.515597 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:22.516038 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:22.516049 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:22.517814 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":82,"threads_started":1}
W20260807 22:43:22.518045 9469 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:22.512573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:41255 (local address 127.26.189.254:36329)
0807 22:43:22.512689 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:22.512692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:22.512778 (+ 86us) server_negotiation.cc:407] Connection header received
0807 22:43:22.512880 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:22.512882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:22.512925 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:22.512994 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:22.513378 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.514005 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.515100 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:22.515403 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:22.515483 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:22.517942 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:41255: BlockingRecv error: recv got EOF from 127.26.189.195:41255 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 22:43:23 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:23 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:23.134496 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:23.129421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:38019)
0807 22:43:23.129710 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:23.129723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:23.129816 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:23.130021 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:23.130023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:23.130194 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:23.130313 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:23.130320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.131014 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.131016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:23.131554 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:23.131560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.131933 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.131935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:23.131993 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.132442 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:23.132453 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:23.134373 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":71,"threads_started":1}
W20260807 22:43:23.134624 9475 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:23.129478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:38019 (local address 127.26.189.254:36329)
0807 22:43:23.129710 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:23.129714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:23.129780 (+ 66us) server_negotiation.cc:407] Connection header received
0807 22:43:23.129883 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:23.129885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:23.129928 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:23.130000 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:23.130403 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.130940 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.131647 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.131857 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.131923 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.134505 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:38019: BlockingRecv error: recv got EOF from 127.26.189.193:38019 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":64,"threads_started":1}
Aug 07 22:43:23 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:23 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:23.335671 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:23.330279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:44851)
0807 22:43:23.330570 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:23.330583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:23.330701 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:23.330939 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:23.330942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:23.331160 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:23.331264 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:23.331270 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.332006 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.332010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:23.332566 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:23.332574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.332996 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.333000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:23.333086 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.333545 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:23.333555 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:23.335512 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":73,"threads_started":1}
W20260807 22:43:23.335748 9475 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:23.330366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:44851 (local address 127.26.189.254:36329)
0807 22:43:23.330498 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:23.330503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:23.330652 (+ 149us) server_negotiation.cc:407] Connection header received
0807 22:43:23.330785 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:23.330788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:23.330834 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:23.330901 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:23.331375 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.331902 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.332672 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.332894 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.332977 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.335649 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:44851: BlockingRecv error: recv got EOF from 127.26.189.194:44851 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 22:43:23 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:23 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:23.523749 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:23.518700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:48113)
0807 22:43:23.518973 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:23.518986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:23.519094 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:23.519330 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:23.519333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:23.519526 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:23.519623 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:23.519629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.520357 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.520361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:23.520914 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:23.520922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.521356 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.521360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:23.521441 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.521896 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:23.521907 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:23.523604 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":71,"threads_started":1}
W20260807 22:43:23.523847 9475 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:23.518772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:48113 (local address 127.26.189.254:36329)
0807 22:43:23.518897 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:23.518902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:23.519050 (+ 148us) server_negotiation.cc:407] Connection header received
0807 22:43:23.519183 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:23.519186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:23.519233 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:23.519296 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:23.519731 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.520254 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.521030 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:23.521252 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:23.521338 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:23.523746 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:48113: BlockingRecv error: recv got EOF from 127.26.189.195:48113 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 22:43:24 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:24 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:24.140650 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:24.135244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:53049)
0807 22:43:24.135529 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:24.135543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:24.135656 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:24.135874 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:24.135876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:24.136056 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:24.136151 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.136157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.136899 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.136902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.137466 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.137474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.137954 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.137956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.138013 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.138499 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:24.138510 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:24.140503 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
W20260807 22:43:24.140720 9482 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:24.135314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:53049 (local address 127.26.189.254:36329)
0807 22:43:24.135569 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:24.135572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:24.135589 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:43:24.135731 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:24.135733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:24.135775 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:24.135848 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:24.136230 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.136810 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.137597 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.137846 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.137952 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.140615 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:53049: BlockingRecv error: recv got EOF from 127.26.189.193:53049 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":94,"threads_started":1}
Aug 07 22:43:24 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:24 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:24.341564 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:24.336396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:51003)
0807 22:43:24.336708 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:24.336724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:24.336833 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:24.337074 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:24.337077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:24.337287 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:24.337385 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.337391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.338130 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.338134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.338724 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.338732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.339160 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.339165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.339246 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.339691 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:24.339702 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:24.341412 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":87,"threads_started":1}
W20260807 22:43:24.341666 9482 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:24.336553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:51003 (local address 127.26.189.254:36329)
0807 22:43:24.336701 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:24.336706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:24.336791 (+ 85us) server_negotiation.cc:407] Connection header received
0807 22:43:24.336918 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:24.336921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:24.336969 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:24.337033 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:24.337501 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.338023 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.338836 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.339057 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.339151 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.341561 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:51003: BlockingRecv error: recv got EOF from 127.26.189.194:51003 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 22:43:24 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:24 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:24.529606 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:24.524575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:54293)
0807 22:43:24.524860 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:24.524872 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:24.524975 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:24.525224 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:24.525227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:24.525422 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:24.525517 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.525523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.526262 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.526266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.526841 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:24.526849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.527274 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.527278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:24.527359 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.527796 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:24.527807 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:24.529462 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":95,"threads_started":1}
W20260807 22:43:24.529721 9482 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:24.524652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:54293 (local address 127.26.189.254:36329)
0807 22:43:24.524794 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:24.524798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:24.524940 (+ 142us) server_negotiation.cc:407] Connection header received
0807 22:43:24.525058 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:24.525062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:24.525111 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:24.525175 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:24.525640 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.526159 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.526950 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:24.527175 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:24.527261 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:24.529596 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:54293: BlockingRecv error: recv got EOF from 127.26.189.195:54293 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 22:43:25 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:25 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:25.146581 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.141418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:48195)
0807 22:43:25.141714 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.141727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.141827 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.142043 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.142045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.142226 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:25.142342 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.142348 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.143120 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.143122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.143679 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.143685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.144061 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.144063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.144115 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.144669 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:25.144681 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:25.146466 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":82,"threads_started":1}
W20260807 22:43:25.146718 9488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:25.141419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:48195 (local address 127.26.189.254:36329)
0807 22:43:25.141680 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.141684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.141786 (+ 102us) server_negotiation.cc:407] Connection header received
0807 22:43:25.141908 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.141910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.141955 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.142022 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:25.142455 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.143036 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.143763 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.143982 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.144052 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.146586 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:48195: BlockingRecv error: recv got EOF from 127.26.189.193:48195 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":88,"threads_started":1}
Aug 07 22:43:25 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:25 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:25.347580 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.342401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:58299)
0807 22:43:25.342681 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.342695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.342793 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.343003 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.343005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.343195 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:25.343297 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.343302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.344010 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.344012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.344569 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.344577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.344980 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.344983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.345052 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.345509 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:25.345520 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:25.347430 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":87,"threads_started":1}
W20260807 22:43:25.347793 9488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:25.342443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:58299 (local address 127.26.189.254:36329)
0807 22:43:25.342565 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.342570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.342750 (+ 180us) server_negotiation.cc:407] Connection header received
0807 22:43:25.342873 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.342875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.342921 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.342977 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:25.343378 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.343908 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.344658 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.344886 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.344950 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.347578 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:58299: BlockingRecv error: recv got EOF from 127.26.189.194:58299 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 22:43:25 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:25 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:25.535601 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:25.530439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:37807)
0807 22:43:25.530780 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:25.530794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:25.530911 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:25.531120 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:25.531122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:25.531300 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:25.531397 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.531403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.532215 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.532218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.532772 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:25.532780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.533211 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.533214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:25.533296 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.533729 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:25.533740 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:25.535480 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":101,"threads_started":1}
W20260807 22:43:25.535709 9488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:25.530480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:37807 (local address 127.26.189.254:36329)
0807 22:43:25.530628 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:25.530633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:25.530852 (+ 219us) server_negotiation.cc:407] Connection header received
0807 22:43:25.530994 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:25.530996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:25.531043 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:25.531108 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:25.531526 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.532088 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.532875 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:25.533101 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:25.533194 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:25.535607 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:37807: BlockingRecv error: recv got EOF from 127.26.189.195:37807 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 22:43:26 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:26 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:26.152652 9492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.147369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:35227)
0807 22:43:26.147811 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.147824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:26.147922 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:26.148139 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:26.148141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:26.148317 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:26.148414 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.148420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.149109 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.149113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.149669 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.149677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.150073 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.150077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.150147 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.150628 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:26.150640 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:26.152477 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":84,"threads_started":1}
W20260807 22:43:26.152732 9493 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:26.147417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:35227 (local address 127.26.189.254:36329)
0807 22:43:26.147695 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.147699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.147887 (+ 188us) server_negotiation.cc:407] Connection header received
0807 22:43:26.148000 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.148002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.148048 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.148118 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:26.148496 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.149020 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.149751 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.149960 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.150027 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.152617 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:35227: BlockingRecv error: recv got EOF from 127.26.189.193:35227 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
Aug 07 22:43:26 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:26 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:26.353446 9494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.348245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:35627)
0807 22:43:26.348597 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.348611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:26.348723 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:26.348966 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:26.348969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:26.349177 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:26.349275 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.349281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.350027 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.350031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.350642 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.350652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.351073 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.351077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.351156 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.351623 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:26.351634 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:26.353300 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":86,"threads_started":1}
W20260807 22:43:26.353542 9493 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:26.348347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:35627 (local address 127.26.189.254:36329)
0807 22:43:26.348492 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.348497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.348682 (+ 185us) server_negotiation.cc:407] Connection header received
0807 22:43:26.348810 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.348814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.348859 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.348929 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:26.349393 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.349921 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.350752 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.350971 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.351056 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.353443 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:35627: BlockingRecv error: recv got EOF from 127.26.189.194:35627 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 22:43:26 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:26 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:26.544160 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:26.536357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:56299)
0807 22:43:26.536675 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:26.536688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:26.536789 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:26.537001 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:26.537003 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:26.537185 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:26.537281 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.537286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.537971 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.537973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.538549 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:26.538556 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.538950 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.538953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:26.539008 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.542171 (+ 3163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:26.542184 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:26.544041 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":80,"threads_started":1}
W20260807 22:43:26.544313 9493 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:26.536432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:56299 (local address 127.26.189.254:36329)
0807 22:43:26.536586 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:26.536591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:26.536754 (+ 163us) server_negotiation.cc:407] Connection header received
0807 22:43:26.536858 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:26.536860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:26.536909 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:26.536973 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:26.537362 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.537890 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.538640 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:26.538869 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:26.538942 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:26.544191 (+ 5249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:56299: BlockingRecv error: recv got EOF from 127.26.189.195:56299 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 22:43:27 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:27 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:27.160665 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:27.153529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:44119)
0807 22:43:27.153904 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:27.153919 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:27.154032 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:27.154318 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:27.154321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:27.154522 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:27.154643 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:27.154649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.155687 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.155692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:27.156638 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:27.156653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.157394 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.157401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:27.157501 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:27.158107 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:27.158125 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:27.160510 (+ 2385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":103,"threads_started":1}
W20260807 22:43:27.160830 9498 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:27.153682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:44119 (local address 127.26.189.254:36329)
0807 22:43:27.153906 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:27.153911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:27.153972 (+ 61us) server_negotiation.cc:407] Connection header received
0807 22:43:27.154131 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:27.154134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:27.154181 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:27.154253 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:27.154755 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.155523 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.156776 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.157136 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.157247 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:27.160677 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:44119: BlockingRecv error: recv got EOF from 127.26.189.193:44119 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":66,"threads_started":1}
Aug 07 22:43:27 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:27 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:27.359688 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:27.354253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:52441)
0807 22:43:27.354532 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:27.354546 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:27.354647 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:27.354848 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:27.354850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:27.355022 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:27.355135 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:27.355140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.355962 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.355964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:27.356551 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:27.356558 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.356933 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.356936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:27.357002 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:27.357455 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:27.357465 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:27.359536 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":105,"threads_started":1}
W20260807 22:43:27.359797 9498 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:27.354337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:52441 (local address 127.26.189.254:36329)
0807 22:43:27.354481 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:27.354486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:27.354610 (+ 124us) server_negotiation.cc:407] Connection header received
0807 22:43:27.354715 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:27.354717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:27.354760 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:27.354824 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:27.355214 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.355874 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.356637 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.356849 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.356916 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:27.359699 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:52441: BlockingRecv error: recv got EOF from 127.26.189.194:52441 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 22:43:27 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:27 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:27.550127 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:27.544862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:37001)
0807 22:43:27.545121 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:27.545134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:27.545260 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:27.545478 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:27.545480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:27.545699 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:27.545818 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:27.545825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.546630 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.546633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:27.547244 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:27.547255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.547710 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.547712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:27.547769 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:27.548218 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:27.548228 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:27.549981 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
W20260807 22:43:27.550280 9498 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:27.544909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:37001 (local address 127.26.189.254:36329)
0807 22:43:27.545031 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:27.545034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:27.545193 (+ 159us) server_negotiation.cc:407] Connection header received
0807 22:43:27.545336 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:27.545338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:27.545388 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:27.545453 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:27.545947 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.546539 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.547352 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:27.547622 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:27.547704 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:27.550138 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:37001: BlockingRecv error: recv got EOF from 127.26.189.195:37001 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 22:43:28 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:28 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:28.166505 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.161382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:54977)
0807 22:43:28.161695 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:28.161708 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:28.161810 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:28.162036 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:28.162038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:28.162219 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:28.162351 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:28.162357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.163052 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.163055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:28.163620 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:28.163628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.164023 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.164026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:28.164088 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:28.164554 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:28.164566 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:28.166390 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":81,"threads_started":1}
W20260807 22:43:28.166628 9512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:28.161453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:54977 (local address 127.26.189.254:36329)
0807 22:43:28.161704 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:28.161708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:28.161761 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:43:28.161882 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:28.161884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:28.161928 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:28.162002 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:28.162438 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.162962 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.163715 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.163937 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.164008 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:28.166514 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:54977: BlockingRecv error: recv got EOF from 127.26.189.193:54977 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
Aug 07 22:43:28 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:28 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:28.366057 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.360372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:37257)
0807 22:43:28.360711 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:28.360727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:28.360847 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:28.361104 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:28.361106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:28.361312 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:28.361419 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:28.361425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.362193 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.362197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:28.362780 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:28.362790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.363238 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.363242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:28.363329 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:28.363874 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:28.363889 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:28.365883 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":90,"threads_started":1}
W20260807 22:43:28.366135 9512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:28.360473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:37257 (local address 127.26.189.254:36329)
0807 22:43:28.360672 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:28.360677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:28.360800 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:43:28.360937 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:28.360940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:28.360989 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:28.361063 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:28.361543 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.362083 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.362899 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.363130 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.363219 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:28.366033 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:37257: BlockingRecv error: recv got EOF from 127.26.189.194:37257 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 22:43:28 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:28 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:28.556175 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:28.550979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:52213)
0807 22:43:28.551255 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:28.551269 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:28.551368 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:28.551621 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:28.551623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:28.551799 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:28.551897 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:28.551901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.552629 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.552631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:28.553224 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:28.553232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.553743 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.553746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:28.553810 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:28.554250 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:28.554260 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:28.556022 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
W20260807 22:43:28.556277 9512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:28.551058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:52213 (local address 127.26.189.254:36329)
0807 22:43:28.551201 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:28.551205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:28.551355 (+ 150us) server_negotiation.cc:407] Connection header received
0807 22:43:28.551435 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:28.551438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:28.551486 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:28.551588 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:28.552009 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.552550 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.553351 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:28.553635 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:28.553733 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:28.556181 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:52213: BlockingRecv error: recv got EOF from 127.26.189.195:52213 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 22:43:29 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:29 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:29.172523 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.167294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:38777)
0807 22:43:29.167551 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.167565 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.167651 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.167866 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.167868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.168080 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.168186 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:29.168192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.168922 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.168925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:29.169498 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:29.169506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.169889 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.169892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:29.169949 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:29.170437 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:29.170448 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.172361 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":71,"threads_started":1}
W20260807 22:43:29.172667 9517 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:29.167377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:38777 (local address 127.26.189.254:36329)
0807 22:43:29.167627 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.167630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.167643 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:29.167725 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.167727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.167769 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.167842 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.168282 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.168828 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.169586 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.169813 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.169880 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:29.172503 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:38777: BlockingRecv error: recv got EOF from 127.26.189.193:38777 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":71,"threads_started":1}
Aug 07 22:43:29 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:29 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:29.372275 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.366977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:43559)
0807 22:43:29.367282 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.367295 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.367391 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.367614 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.367616 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.367803 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.367902 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:29.367908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.368613 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.368615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:29.369169 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:29.369177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.369560 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.369563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:29.369643 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:29.370092 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:29.370103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.372121 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
W20260807 22:43:29.372349 9517 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:29.367047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:43559 (local address 127.26.189.254:36329)
0807 22:43:29.367195 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.367200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.367358 (+ 158us) server_negotiation.cc:407] Connection header received
0807 22:43:29.367463 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.367465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.367510 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.367577 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.367986 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.368514 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.369254 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.369475 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.369544 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:29.372253 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:43559: BlockingRecv error: recv got EOF from 127.26.189.194:43559 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 22:43:29 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:29 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:29.562063 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:29.556958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:57445)
0807 22:43:29.557239 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:29.557254 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:29.557362 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:29.557601 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:29.557604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:29.557816 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:29.557927 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:29.557933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.558758 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.558762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:29.559317 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:29.559325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.559754 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.559758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:29.559842 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:29.560276 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:29.560287 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:29.561947 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
W20260807 22:43:29.562177 9517 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:29.557100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:57445 (local address 127.26.189.254:36329)
0807 22:43:29.557239 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:29.557243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:29.557320 (+ 77us) server_negotiation.cc:407] Connection header received
0807 22:43:29.557447 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:29.557450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:29.557501 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:29.557565 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:29.558043 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.558651 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.559431 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:29.559656 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:29.559739 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:29.562077 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:57445: BlockingRecv error: recv got EOF from 127.26.189.195:57445 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 22:43:30 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:30 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:30.178643 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:30.173286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:58427)
0807 22:43:30.173697 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:30.173710 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:30.173806 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:30.174074 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:30.174076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:30.174355 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:30.174459 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.174465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.175233 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.175237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.175787 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.175794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.176221 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.176224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.176304 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.176761 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:30.176771 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:30.178521 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
W20260807 22:43:30.178788 9521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:30.173379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:58427 (local address 127.26.189.254:36329)
0807 22:43:30.173668 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:30.173672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:30.173858 (+ 186us) server_negotiation.cc:407] Connection header received
0807 22:43:30.173901 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:30.173904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:30.173959 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:30.174048 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:30.174576 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.175127 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.175896 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.176123 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.176206 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.178665 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:58427: BlockingRecv error: recv got EOF from 127.26.189.193:58427 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
Aug 07 22:43:30 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:30 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:30.378448 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:30.373042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:50999)
0807 22:43:30.373340 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:30.373355 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:30.373469 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:30.373725 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:30.373728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:30.373929 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:30.374031 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.374038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.374854 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.374858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.375423 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.375431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.375870 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.375875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.375963 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.376428 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:30.376439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:30.378273 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":90,"threads_started":1}
W20260807 22:43:30.378536 9521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:30.373145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:50999 (local address 127.26.189.254:36329)
0807 22:43:30.373307 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:30.373312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:30.373420 (+ 108us) server_negotiation.cc:407] Connection header received
0807 22:43:30.373550 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:30.373554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:30.373614 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:30.373682 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:30.374145 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.374746 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.375531 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.375765 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.375853 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.378434 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:50999: BlockingRecv error: recv got EOF from 127.26.189.194:50999 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 22:43:30 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:30 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:30.568002 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:30.562902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:33965)
0807 22:43:30.563217 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:30.563234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:30.563345 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:30.563571 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:30.563574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:30.563765 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:30.563873 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.563879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.564571 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.564574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.565138 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:30.565147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.565542 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.565545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:30.565610 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.566077 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:30.566089 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:30.567868 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":123,"threads_started":1}
W20260807 22:43:30.568099 9521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:30.563013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:33965 (local address 127.26.189.254:36329)
0807 22:43:30.563133 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:30.563137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:30.563289 (+ 152us) server_negotiation.cc:407] Connection header received
0807 22:43:30.563421 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:30.563423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:30.563466 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:30.563534 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:30.563960 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.564469 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.565242 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:30.565450 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:30.565516 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:30.568002 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:33965: BlockingRecv error: recv got EOF from 127.26.189.195:33965 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 22:43:31 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:31 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:31.184373 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:31.179377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:39079)
0807 22:43:31.179650 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:31.179663 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:31.179754 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:31.179955 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:31.179957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:31.180144 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:31.180241 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.180246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.180953 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.180955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.181525 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.181532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.181915 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.181918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.181973 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.182455 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:31.182466 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:31.184229 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":72,"threads_started":1}
W20260807 22:43:31.184482 9527 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:31.179447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:39079 (local address 127.26.189.254:36329)
0807 22:43:31.179715 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:31.179719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:31.179732 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:43:31.179819 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:31.179821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:31.179861 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:31.179932 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:31.180327 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.180878 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.181622 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.181839 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.181905 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.184368 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:39079: BlockingRecv error: recv got EOF from 127.26.189.193:39079 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
Aug 07 22:43:31 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:31 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:31.385739 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:31.379437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:56095)
0807 22:43:31.379713 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:31.379728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:31.379846 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:31.380126 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:31.380128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:31.380318 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:31.380442 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.380450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.381459 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.381463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.382250 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.382262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.382760 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.382764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.382851 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.383356 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:31.383371 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:31.385586 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
W20260807 22:43:31.385890 9527 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:31.379521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:56095 (local address 127.26.189.254:36329)
0807 22:43:31.379676 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:31.379681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:31.379786 (+ 105us) server_negotiation.cc:407] Connection header received
0807 22:43:31.379939 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:31.379942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:31.379998 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:31.380069 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:31.380547 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.381344 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.382397 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.382649 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.382737 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.385768 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:56095: BlockingRecv error: recv got EOF from 127.26.189.194:56095 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 22:43:31 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:31 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:31.577618 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:31.568750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:39033)
0807 22:43:31.569020 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:31.569033 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:31.569144 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:31.569378 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:31.569381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:31.569572 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:31.569690 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.569697 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.570534 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.570537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.571104 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:31.571112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.571535 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.571539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:31.571630 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.575772 (+ 4142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:31.575785 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:31.577492 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
W20260807 22:43:31.577721 9527 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:31.568845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:39033 (local address 127.26.189.254:36329)
0807 22:43:31.568988 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:31.568993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:31.569102 (+ 109us) server_negotiation.cc:407] Connection header received
0807 22:43:31.569227 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:31.569230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:31.569278 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:31.569340 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:31.569803 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.570425 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.571213 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:31.571433 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:31.571519 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:31.577624 (+ 6105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:39033: BlockingRecv error: recv got EOF from 127.26.189.195:39033 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 22:43:32 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:32 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:32.190685 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:32.185268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:41647)
0807 22:43:32.185540 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:32.185555 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:32.185665 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:32.185878 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:32.185879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:32.186068 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:32.186166 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.186171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.186927 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.186931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.187662 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.187677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.188132 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.188136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.188210 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.188716 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:32.188728 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:32.190494 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":69,"threads_started":1}
W20260807 22:43:32.190747 9531 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:32.185326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:41647 (local address 127.26.189.254:36329)
0807 22:43:32.185570 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:32.185573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:32.185612 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:43:32.185738 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:32.185740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:32.185783 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:32.185856 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:32.186251 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.186813 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.187790 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.188019 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.188121 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.190633 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:41647: BlockingRecv error: recv got EOF from 127.26.189.193:41647 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
Aug 07 22:43:32 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:32 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:32.391815 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:32.386560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:41023)
0807 22:43:32.386854 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:32.386870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:32.386983 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:32.387226 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:32.387229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:32.387437 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:32.387537 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.387543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.388288 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.388292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.388856 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.388864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.389301 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.389305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.389393 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.389838 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:32.389848 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:32.391668 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
W20260807 22:43:32.391907 9531 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:32.386715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:41023 (local address 127.26.189.254:36329)
0807 22:43:32.386856 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:32.386861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:32.386926 (+ 65us) server_negotiation.cc:407] Connection header received
0807 22:43:32.387070 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:32.387073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:32.387122 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:32.387189 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:32.387662 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.388185 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.388965 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.389198 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.389284 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.391805 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:41023: BlockingRecv error: recv got EOF from 127.26.189.194:41023 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 22:43:32 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:32 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:32.583590 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:32.578394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:41679)
0807 22:43:32.578646 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:32.578658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:32.578758 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:32.578953 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:32.578955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:32.579143 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:32.579241 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.579247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.579991 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.579994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.580592 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:32.580601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.580965 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.580968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:32.581043 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.581507 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:32.581523 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:32.583414 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
W20260807 22:43:32.583703 9531 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:32.578468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:41679 (local address 127.26.189.254:36329)
0807 22:43:32.578594 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:32.578598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:32.578718 (+ 120us) server_negotiation.cc:407] Connection header received
0807 22:43:32.578826 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:32.578829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:32.578873 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:32.578942 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:32.579320 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.579857 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.580686 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:32.580889 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:32.580952 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:32.583577 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:41679: BlockingRecv error: recv got EOF from 127.26.189.195:41679 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 22:43:33 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:33 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:33.196825 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:33.191432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:33981)
0807 22:43:33.191728 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:33.191745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:33.191852 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:33.192062 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:33.192064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:33.192243 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:33.192339 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.192345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.193063 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.193066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.193644 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.193653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.194066 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.194069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.194133 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.194703 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:33.194716 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:33.196710 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
W20260807 22:43:33.196938 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:33.191563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:33981 (local address 127.26.189.254:36329)
0807 22:43:33.191878 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:33.191882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:33.191894 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:43:33.191927 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:33.191929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:33.191969 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:33.192040 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:33.192422 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.192963 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.193739 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.193960 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.194032 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.196827 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:33981: BlockingRecv error: recv got EOF from 127.26.189.193:33981 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":82,"threads_started":1}
Aug 07 22:43:33 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:33 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:33.397943 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:33.392590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:49557)
0807 22:43:33.392885 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:33.392899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:33.393011 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:33.393251 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:33.393254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:33.393451 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:33.393598 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.393605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.394328 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.394331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.395004 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.395016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.395410 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.395413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.395503 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.395968 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:33.395980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:33.397810 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":84,"threads_started":1}
W20260807 22:43:33.398034 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:33.392659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:49557 (local address 127.26.189.254:36329)
0807 22:43:33.392785 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:33.392788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:33.392965 (+ 177us) server_negotiation.cc:407] Connection header received
0807 22:43:33.393086 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:33.393088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:33.393132 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:33.393199 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:33.393698 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.394201 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.395109 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.395322 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.395397 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.397943 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:49557: BlockingRecv error: recv got EOF from 127.26.189.194:49557 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 22:43:33 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:33 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.195@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:33.589650 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:33.584431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.195:53409)
0807 22:43:33.584815 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:33.584828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:33.584954 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:33.585253 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:33.585256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:33.585470 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:33.585575 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.585581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.586329 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.586331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.586915 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:33.586923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.587311 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.587313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:33.587370 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.587818 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:33.587828 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:33.589524 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":133,"threads_started":1}
W20260807 22:43:33.589776 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:33.584546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:53409 (local address 127.26.189.254:36329)
0807 22:43:33.584679 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:33.584684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:33.584924 (+ 240us) server_negotiation.cc:407] Connection header received
0807 22:43:33.585052 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:33.585055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:33.585129 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:33.585194 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:33.585675 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.586204 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.587013 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:33.587234 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:33.587304 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:33.589655 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.195:53409: BlockingRecv error: recv got EOF from 127.26.189.195:53409 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 22:43:34 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:34 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.193@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:34.202919 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:34.197708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.193:55795)
0807 22:43:34.197973 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:34.197986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:34.198078 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:34.198330 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:34.198333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:34.198521 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:34.198623 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:34.198629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.199306 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.199309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:34.199849 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:34.199856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.200197 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.200199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:34.200256 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.200709 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:34.200719 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:34.202757 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":71,"threads_started":1}
W20260807 22:43:34.203022 9542 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:34.197772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:55795 (local address 127.26.189.254:36329)
0807 22:43:34.197995 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:34.197998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:34.198035 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:43:34.198165 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:34.198167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:34.198209 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:34.198282 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:34.198707 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.199227 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.199920 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.200125 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.200188 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.202902 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.193:55795: BlockingRecv error: recv got EOF from 127.26.189.193:55795 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
Aug 07 22:43:34 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:43:34 dist-test-slave-4tdj krb5kdc[7900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.189.194@KRBTEST.COM for oryx/127.26.189.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:43:34.404059 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:34.398843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36329 (local address 127.26.189.194:55897)
0807 22:43:34.399122 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:34.399137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:34.399251 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:34.399534 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:34.399537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:34.399731 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:34.399827 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:34.399832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.400591 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.400595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:34.401146 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:34.401153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.401583 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.401587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:34.401667 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.402161 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:34.402173 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:34.403914 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.189.254:36329: Server oryx/127.26.189.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
W20260807 22:43:34.404150 9542 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:34.398956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:55897 (local address 127.26.189.254:36329)
0807 22:43:34.399124 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:34.399130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:34.399209 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:43:34.399337 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:34.399341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:34.399388 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:34.399453 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:34.399944 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.400484 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.401255 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.401481 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.401568 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.404050 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.189.194:55897: BlockingRecv error: recv got EOF from 127.26.189.194:55897 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 22:43:34.483180 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 8798
I20260807 22:43:34.497025 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 9045
I20260807 22:43:34.507632 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 9237
I20260807 22:43:34.517102 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 8704
2026-08-07T22:43:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18085 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T22:43:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:34Z Disabled control of system clock
I20260807 22:43:34.551085 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38809
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:43:34.659101 9548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:34.659333 9548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:34.659369 9548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:34.663313 9548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:34.663362 9548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:34.663378 9548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:34.663407 9548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:34.663434 9548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:34.668066 9548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:34.669019 9548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:34.670343 9548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:43:34.678447 9548 server_base.cc:1034] running on GCE node
W20260807 22:43:34.678455 9566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:34.678507 9560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:34.678606 9561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:34.679244 9548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:34.680281 9548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:34.681443 9548 hybrid_clock.cc:648] HybridClock initialized: now 1786142614681430 us; error 24 us; skew 500 ppm
I20260807 22:43:34.683193 9548 webserver.cc:460] Webserver started at http://127.26.189.254:43739/ using document root <none> and password file <none>
I20260807 22:43:34.683722 9548 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:34.683789 9548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:34.683982 9548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:34.686008 9548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "b377f46c0fdd4d53a61c2af3f87f7328"
format_stamp: "Formatted at 2026-08-07 22:43:34 on dist-test-slave-4tdj"
server_key: "95c04ea93de97f4da1af6d5ea0682759"
server_key_iv: "7e09fd319c354f197ba4b72d093bb41b"
server_key_version: "encryptionkey@0"
I20260807 22:43:34.686614 9548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "b377f46c0fdd4d53a61c2af3f87f7328"
format_stamp: "Formatted at 2026-08-07 22:43:34 on dist-test-slave-4tdj"
server_key: "95c04ea93de97f4da1af6d5ea0682759"
server_key_iv: "7e09fd319c354f197ba4b72d093bb41b"
server_key_version: "encryptionkey@0"
I20260807 22:43:34.690049 9548 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:34.692312 9575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:34.693444 9548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:34.693565 9548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "b377f46c0fdd4d53a61c2af3f87f7328"
format_stamp: "Formatted at 2026-08-07 22:43:34 on dist-test-slave-4tdj"
server_key: "95c04ea93de97f4da1af6d5ea0682759"
server_key_iv: "7e09fd319c354f197ba4b72d093bb41b"
server_key_version: "encryptionkey@0"
I20260807 22:43:34.693670 9548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:34.704130 9548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:34.704725 9548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:34.704901 9548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:34.711479 9548 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38809
I20260807 22:43:34.711494 9653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38809 every 8 connection(s)
I20260807 22:43:34.712345 9548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:34.715412 9654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:34.716392 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 9548
I20260807 22:43:34.716496 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:34.716846 27383 external_mini_cluster.cc:1428] Setting key bfea648317c355678b8547748a420d73
I20260807 22:43:34.721383 9654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328: Bootstrap starting.
I20260807 22:43:34.723557 9654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:34.724264 9654 log.cc:824] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:34.725597 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:34.718242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.26.189.254:38809)
0807 22:43:34.718870 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:34.718885 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:34.718919 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:43:34.719548 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:34.719563 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:34.719930 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:34.720237 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:34.720651 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.721710 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.722609 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.723060 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.723221 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.723978 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:34.723983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:34.723995 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:34.724357 (+ 362us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:34.724424 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:34.724446 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:34.724529 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:34.724685 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:43:34.724931 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":120,"threads_started":1}
I20260807 22:43:34.726071 9654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328: No bootstrap required, opened a new log
I20260807 22:43:34.729457 9654 raft_consensus.cc:348] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38809 } }
I20260807 22:43:34.729672 9654 raft_consensus.cc:374] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:34.729728 9654 raft_consensus.cc:729] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b377f46c0fdd4d53a61c2af3f87f7328, State: Initialized, Role: FOLLOWER
I20260807 22:43:34.730152 9654 consensus_queue.cc:260] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38809 } }
I20260807 22:43:34.730306 9654 raft_consensus.cc:388] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:34.730388 9654 raft_consensus.cc:482] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:34.730490 9654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:34.731302 9654 raft_consensus.cc:504] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38809 } }
I20260807 22:43:34.731611 9654 leader_election.cc:302] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b377f46c0fdd4d53a61c2af3f87f7328; no voters:
I20260807 22:43:34.731876 9654 leader_election.cc:288] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:34.731974 9659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:34.732196 9659 raft_consensus.cc:686] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [term 1 LEADER]: Becoming Leader. State: Replica: b377f46c0fdd4d53a61c2af3f87f7328, State: Running, Role: LEADER
I20260807 22:43:34.732540 9659 consensus_queue.cc:237] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38809 } }
I20260807 22:43:34.732993 9654 sys_catalog.cc:564] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:34.734143 9661 sys_catalog.cc:455] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b377f46c0fdd4d53a61c2af3f87f7328. Latest consensus state: current_term: 1 leader_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38809 } } }
I20260807 22:43:34.734191 9660 sys_catalog.cc:455] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b377f46c0fdd4d53a61c2af3f87f7328" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38809 } } }
I20260807 22:43:34.734259 9661 sys_catalog.cc:458] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:34.734283 9660 sys_catalog.cc:458] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:34.734661 9676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:34.737588 9676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:34.742710 9676 catalog_manager.cc:1347] Generated new cluster ID: c2ed00ba3cab48e1b314e10d8a37409f
I20260807 22:43:34.742771 9676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:34.758744 9676 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:34.759562 9676 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:34.769761 9676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b377f46c0fdd4d53a61c2af3f87f7328: Generated new TSK 0
I20260807 22:43:34.770401 9676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:34.779758 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38809
--builtin_ntp_servers=127.26.189.212:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 22:43:34.889669 9691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:34.889899 9691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:34.889933 9691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:34.893847 9691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:34.893898 9691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:34.893949 9691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:34.898514 9691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:34.899439 9691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:34.900738 9691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:34.909886 9703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:34.909888 9705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:34.909883 9702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:34.910049 9691 server_base.cc:1034] running on GCE node
I20260807 22:43:34.910583 9691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:34.911170 9691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:34.912367 9691 hybrid_clock.cc:648] HybridClock initialized: now 1786142614912348 us; error 28 us; skew 500 ppm
I20260807 22:43:34.914283 9691 webserver.cc:460] Webserver started at http://127.26.189.193:33247/ using document root <none> and password file <none>
I20260807 22:43:34.914852 9691 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:34.914915 9691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:34.915128 9691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:34.917107 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "8c57b440d3e8403cb0e286e0f0b53360"
format_stamp: "Formatted at 2026-08-07 22:43:34 on dist-test-slave-4tdj"
server_key: "915cd790a60dc94ddd3a8147da25c0d7"
server_key_iv: "d40ef39d331d41e930e7a82239408d87"
server_key_version: "encryptionkey@0"
I20260807 22:43:34.917630 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "8c57b440d3e8403cb0e286e0f0b53360"
format_stamp: "Formatted at 2026-08-07 22:43:34 on dist-test-slave-4tdj"
server_key: "915cd790a60dc94ddd3a8147da25c0d7"
server_key_iv: "d40ef39d331d41e930e7a82239408d87"
server_key_version: "encryptionkey@0"
I20260807 22:43:34.921176 9691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:43:34.923667 9714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:34.924793 9691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:34.924916 9691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "8c57b440d3e8403cb0e286e0f0b53360"
format_stamp: "Formatted at 2026-08-07 22:43:34 on dist-test-slave-4tdj"
server_key: "915cd790a60dc94ddd3a8147da25c0d7"
server_key_iv: "d40ef39d331d41e930e7a82239408d87"
server_key_version: "encryptionkey@0"
I20260807 22:43:34.925025 9691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:34.936995 9691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:34.937601 9691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:34.937762 9691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:34.938324 9691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:34.939224 9691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:34.939270 9691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:34.939311 9691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:34.939338 9691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:34.947372 9691 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:44729
I20260807 22:43:34.947427 9873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:44729 every 8 connection(s)
I20260807 22:43:34.948230 9691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:34.956395 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 9691
I20260807 22:43:34.956517 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:34.956712 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:34.949868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:55615 (local address 127.26.189.254:38809)
0807 22:43:34.950034 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:34.950039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:34.951078 (+ 1039us) server_negotiation.cc:407] Connection header received
0807 22:43:34.952391 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:34.952393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:34.952445 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:34.952519 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:34.953341 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.953886 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.954972 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.955188 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.955262 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.956196 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:34.956201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:34.956203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:34.956218 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:34.956239 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:34.956248 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:34.956314 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:34.956481 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:43:34.956549 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:43:34.956854 27383 external_mini_cluster.cc:1428] Setting key bb76fdba8c27e367f710ab6df00feafd
I20260807 22:43:34.957583 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:34.950280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38809 (local address 127.26.189.193:55615)
0807 22:43:34.950939 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:34.950978 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:34.952207 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:34.952678 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:34.952684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:34.952762 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:34.953216 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:34.953228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.953978 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.953980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:34.954881 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:34.954891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:34.955295 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:34.955297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:34.955429 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:34.955942 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:34.955944 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:34.955980 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:34.956038 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:34.956330 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:34.956340 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:34.956346 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:34.956473 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 22:43:34.956770 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":113,"threads_started":1}
I20260807 22:43:34.958709 9874 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38809
I20260807 22:43:34.958858 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38809
--builtin_ntp_servers=127.26.189.212:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:43:34.958913 9874 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:34.959380 9874 heartbeater.cc:511] Master 127.26.189.254:38809 requested a full tablet report, sending...
I20260807 22:43:34.960911 9608 ts_manager.cc:194] Registered new tserver with Master: 8c57b440d3e8403cb0e286e0f0b53360 (127.26.189.193:44729)
I20260807 22:43:34.962255 9608 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.193:55615
W20260807 22:43:35.071877 9877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:35.072105 9877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:35.072141 9877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:35.076155 9877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:35.076206 9877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:35.076267 9877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:35.080822 9877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:35.081745 9877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:35.083070 9877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:35.092237 9892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.092238 9888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.092288 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:35.092487 9877 server_base.cc:1034] running on GCE node
I20260807 22:43:35.092962 9877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:35.093554 9877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:35.094708 9877 hybrid_clock.cc:648] HybridClock initialized: now 1786142615094688 us; error 32 us; skew 500 ppm
I20260807 22:43:35.096415 9877 webserver.cc:460] Webserver started at http://127.26.189.194:36353/ using document root <none> and password file <none>
I20260807 22:43:35.096957 9877 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:35.097034 9877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:35.097214 9877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:35.099159 9877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "8fb93434840d416880ea9e2fca9ecb46"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "ccbb212d4309f4d67217b5ce1e7848b6"
server_key_iv: "bdd264dc5a63ed609b27426724e29769"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.099673 9877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "8fb93434840d416880ea9e2fca9ecb46"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "ccbb212d4309f4d67217b5ce1e7848b6"
server_key_iv: "bdd264dc5a63ed609b27426724e29769"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.103084 9877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:35.105346 9904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.106634 9877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 22:43:35.106735 9877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "8fb93434840d416880ea9e2fca9ecb46"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "ccbb212d4309f4d67217b5ce1e7848b6"
server_key_iv: "bdd264dc5a63ed609b27426724e29769"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.106827 9877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:35.124697 9877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:35.125245 9877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:35.125376 9877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:35.125862 9877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:35.126754 9877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:35.126801 9877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.126837 9877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:35.126856 9877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.134907 9877 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:46317
I20260807 22:43:35.134965 10069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:46317 every 8 connection(s)
I20260807 22:43:35.135715 9877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:35.136166 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 9877
I20260807 22:43:35.136281 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:35.136582 27383 external_mini_cluster.cc:1428] Setting key e6910b076923defc583d9fe43452629c
I20260807 22:43:35.138634 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38809
--builtin_ntp_servers=127.26.189.212:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:43:35.145094 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.137336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:35955 (local address 127.26.189.254:38809)
0807 22:43:35.137446 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:35.137449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:35.138611 (+ 1162us) server_negotiation.cc:407] Connection header received
0807 22:43:35.139960 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:35.139963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:35.140015 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:35.140094 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:35.140976 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.141650 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.143261 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.143472 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.143548 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:35.144690 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:35.144693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:35.144695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:35.144706 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:35.144726 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:35.144733 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:35.144785 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:35.144939 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 22:43:35.144986 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:43:35.146191 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.137823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38809 (local address 127.26.189.194:35955)
0807 22:43:35.138487 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:35.138525 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:35.139770 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:35.140320 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:35.140327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:35.140406 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:35.140848 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:35.140860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.141778 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.141781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:35.143150 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:35.143162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.143620 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.143625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:35.143819 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:35.144382 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:35.144384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:35.144438 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:35.144519 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:35.144807 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:35.144816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:35.144822 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:35.144948 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 22:43:35.145243 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":133,"threads_started":1}
I20260807 22:43:35.147250 10071 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38809
I20260807 22:43:35.147452 10071 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:35.147902 10071 heartbeater.cc:511] Master 127.26.189.254:38809 requested a full tablet report, sending...
I20260807 22:43:35.148820 9608 ts_manager.cc:194] Registered new tserver with Master: 8fb93434840d416880ea9e2fca9ecb46 (127.26.189.194:46317)
I20260807 22:43:35.149348 9608 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.194:35955
W20260807 22:43:35.247840 10075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:35.248071 10075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:35.248129 10075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:35.252055 10075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:35.252107 10075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:35.252177 10075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:35.256701 10075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:35.257619 10075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:35.258908 10075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:35.267829 10087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.267863 10085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.267885 10084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:35.268119 10075 server_base.cc:1034] running on GCE node
I20260807 22:43:35.268464 10075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:35.269030 10075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:35.270185 10075 hybrid_clock.cc:648] HybridClock initialized: now 1786142615270154 us; error 46 us; skew 500 ppm
I20260807 22:43:35.271798 10075 webserver.cc:460] Webserver started at http://127.26.189.195:33433/ using document root <none> and password file <none>
I20260807 22:43:35.272324 10075 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:35.272405 10075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:35.272611 10075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:35.274580 10075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "7f97f5e7999a4afba4559d33febf21be"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "1dd756c5cd7e171c6557015e1f2a3028"
server_key_iv: "9a3bff929b3b3bbd877934e9b842b9f9"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.275122 10075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "7f97f5e7999a4afba4559d33febf21be"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "1dd756c5cd7e171c6557015e1f2a3028"
server_key_iv: "9a3bff929b3b3bbd877934e9b842b9f9"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.278409 10075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:43:35.280628 10100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.281713 10075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:35.281816 10075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "7f97f5e7999a4afba4559d33febf21be"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "1dd756c5cd7e171c6557015e1f2a3028"
server_key_iv: "9a3bff929b3b3bbd877934e9b842b9f9"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.281903 10075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:35.294862 10075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:35.295472 10075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:35.295634 10075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:35.296142 10075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:35.296998 10075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:35.297043 10075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.297101 10075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:35.297158 10075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.305101 10075 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:42419
I20260807 22:43:35.305124 10278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:42419 every 8 connection(s)
I20260807 22:43:35.305902 10075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:35.314445 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.307586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:60259 (local address 127.26.189.254:38809)
0807 22:43:35.307736 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:35.307740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:35.308694 (+ 954us) server_negotiation.cc:407] Connection header received
0807 22:43:35.310035 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:35.310038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:35.310093 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:35.310167 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:35.311049 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.311570 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.312666 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.312892 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.312979 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:35.313954 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:35.313958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:35.313961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:35.313974 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:35.313993 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:35.314001 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:35.314059 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:35.314211 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 22:43:35.314272 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:43:35.315186 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 10075
I20260807 22:43:35.315212 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.307935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38809 (local address 127.26.189.195:60259)
0807 22:43:35.308548 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:35.308593 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:35.309828 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:35.310360 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:35.310368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:35.310447 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:35.310911 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:35.310924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.311695 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.311698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:35.312547 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:35.312556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:35.313027 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:35.313030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:35.313198 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:35.313708 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:35.313709 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:35.313748 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:35.313808 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:35.314069 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:35.314078 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:35.314084 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:35.314197 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:43:35.314485 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260807 22:43:35.315279 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:35.315543 27383 external_mini_cluster.cc:1428] Setting key 37fd7cefe7543d364f7d2b7435001a02
I20260807 22:43:35.316396 10280 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38809
I20260807 22:43:35.316640 10280 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:35.317217 10280 heartbeater.cc:511] Master 127.26.189.254:38809 requested a full tablet report, sending...
I20260807 22:43:35.318190 9608 ts_manager.cc:194] Registered new tserver with Master: 7f97f5e7999a4afba4559d33febf21be (127.26.189.195:42419)
I20260807 22:43:35.318747 9608 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.195:60259
I20260807 22:43:35.329079 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:43:35.331875 9657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:35.330954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.26.189.254:38809)
0807 22:43:35.331078 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:35.331083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:35.331263 (+ 180us) server_negotiation.cc:407] Connection header received
0807 22:43:35.331379 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:35.331382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:35.331426 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:35.331489 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:35.331748 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46656: BlockingRecv error: recv got EOF from 127.0.0.1:46656 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:43:35.332190 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 9691
I20260807 22:43:35.362946 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 9877
I20260807 22:43:35.376559 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 10075
I20260807 22:43:35.389317 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 9548
2026-08-07T22:43:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (877 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T22:43:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:35Z Disabled control of system clock
I20260807 22:43:35.427940 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36909
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:42505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:43:35.533574 10295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:35.533799 10295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:35.533833 10295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:35.537678 10295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:35.537727 10295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:35.537741 10295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:35.537756 10295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:35.537772 10295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:35.542246 10295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:35.543133 10295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:35.544353 10295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:35.552225 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:35.552258 10295 server_base.cc:1034] running on GCE node
W20260807 22:43:35.552232 10304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.552284 10306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:35.552945 10295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:35.553949 10295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:35.555109 10295 hybrid_clock.cc:648] HybridClock initialized: now 1786142615555090 us; error 32 us; skew 500 ppm
I20260807 22:43:35.556746 10295 webserver.cc:460] Webserver started at http://127.26.189.254:42955/ using document root <none> and password file <none>
I20260807 22:43:35.557260 10295 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:35.557322 10295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:35.557512 10295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:35.559480 10295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "c7251566396f4cfca642f6bbb3da3e46"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "4dc65b73e6317076b4eb543937fb0816"
server_key_iv: "b91367ae6caa6c0e111735206817549d"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.559999 10295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "c7251566396f4cfca642f6bbb3da3e46"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "4dc65b73e6317076b4eb543937fb0816"
server_key_iv: "b91367ae6caa6c0e111735206817549d"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.563393 10295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:43:35.565558 10321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.566687 10295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:35.566797 10295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "c7251566396f4cfca642f6bbb3da3e46"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "4dc65b73e6317076b4eb543937fb0816"
server_key_iv: "b91367ae6caa6c0e111735206817549d"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.566883 10295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:35.571079 10295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:35.571625 10295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:35.571770 10295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:35.578104 10295 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:36909
I20260807 22:43:35.578107 10395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:36909 every 8 connection(s)
I20260807 22:43:35.578971 10295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:35.581885 10399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:35.582760 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 10295
I20260807 22:43:35.582860 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:35.583125 27383 external_mini_cluster.cc:1428] Setting key 67ec7159cc1b5a5c9ec17e131dd1223c
I20260807 22:43:35.588222 10399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46: Bootstrap starting.
I20260807 22:43:35.588851 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.584398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.26.189.254:36909)
0807 22:43:35.585044 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:35.585064 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:35.585105 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:43:35.585931 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:35.585954 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:35.586418 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:35.586763 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:35.587394 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:35.587399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:35.587413 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:35.587423 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:35.587487 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:35.587509 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:35.587652 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:35.587716 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 22:43:35.587937 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260807 22:43:35.590469 10399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:35.591210 10399 log.cc:824] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:35.592857 10399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46: No bootstrap required, opened a new log
I20260807 22:43:35.596184 10399 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7251566396f4cfca642f6bbb3da3e46" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36909 } }
I20260807 22:43:35.596422 10399 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:35.596481 10399 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7251566396f4cfca642f6bbb3da3e46, State: Initialized, Role: FOLLOWER
I20260807 22:43:35.596901 10399 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7251566396f4cfca642f6bbb3da3e46" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36909 } }
I20260807 22:43:35.597033 10399 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:35.597096 10399 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:35.597196 10399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:35.597972 10399 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7251566396f4cfca642f6bbb3da3e46" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36909 } }
I20260807 22:43:35.598275 10399 leader_election.cc:302] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7251566396f4cfca642f6bbb3da3e46; no voters:
I20260807 22:43:35.598569 10399 leader_election.cc:288] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:35.598663 10408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:35.598856 10408 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [term 1 LEADER]: Becoming Leader. State: Replica: c7251566396f4cfca642f6bbb3da3e46, State: Running, Role: LEADER
I20260807 22:43:35.599164 10408 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7251566396f4cfca642f6bbb3da3e46" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36909 } }
I20260807 22:43:35.599638 10399 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:35.600636 10413 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7251566396f4cfca642f6bbb3da3e46. Latest consensus state: current_term: 1 leader_uuid: "c7251566396f4cfca642f6bbb3da3e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7251566396f4cfca642f6bbb3da3e46" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36909 } } }
I20260807 22:43:35.600761 10413 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:35.601044 10410 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7251566396f4cfca642f6bbb3da3e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7251566396f4cfca642f6bbb3da3e46" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36909 } } }
I20260807 22:43:35.601132 10410 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46 [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:35.601511 10422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:35.604043 10422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:35.609079 10422 catalog_manager.cc:1347] Generated new cluster ID: e0e25459b2274fa1924f86eaa8fad712
I20260807 22:43:35.609146 10422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:35.621021 10422 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:35.621294 10422 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:35.629408 10422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7251566396f4cfca642f6bbb3da3e46: Generated new TSK 0
I20260807 22:43:35.630090 10422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:43:35.639813 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36909
--builtin_ntp_servers=127.26.189.212:42505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:43:35.747661 10437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:35.747892 10437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:35.747946 10437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:35.751942 10437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:35.751989 10437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:35.752063 10437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:35.756634 10437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:35.757599 10437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:35.758937 10437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:35.767868 10447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.768428 10445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.768445 10444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:35.768478 10437 server_base.cc:1034] running on GCE node
I20260807 22:43:35.768865 10437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:35.769479 10437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:35.770643 10437 hybrid_clock.cc:648] HybridClock initialized: now 1786142615770623 us; error 33 us; skew 500 ppm
I20260807 22:43:35.772311 10437 webserver.cc:460] Webserver started at http://127.26.189.193:45003/ using document root <none> and password file <none>
I20260807 22:43:35.772866 10437 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:35.772934 10437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:35.773134 10437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:35.775135 10437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "68d01f803cd94c9d9da9282e5c334ed0"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "775305b30a9ba0a0c190621572ed4ad2"
server_key_iv: "5e7c534887876336f6150e6ceff55898"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.775691 10437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "68d01f803cd94c9d9da9282e5c334ed0"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "775305b30a9ba0a0c190621572ed4ad2"
server_key_iv: "5e7c534887876336f6150e6ceff55898"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.779136 10437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:43:35.781445 10455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.782588 10437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:43:35.782698 10437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "68d01f803cd94c9d9da9282e5c334ed0"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "775305b30a9ba0a0c190621572ed4ad2"
server_key_iv: "5e7c534887876336f6150e6ceff55898"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.782786 10437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:35.787014 10437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:35.787557 10437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:35.787714 10437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:35.788200 10437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:35.789026 10437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:35.789072 10437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.789120 10437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:35.789150 10437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.797134 10437 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:43175
I20260807 22:43:35.797166 10634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:43175 every 8 connection(s)
I20260807 22:43:35.797923 10437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:35.803553 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.799549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37073 (local address 127.26.189.254:36909)
0807 22:43:35.799683 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:35.799687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:35.800704 (+ 1017us) server_negotiation.cc:407] Connection header received
0807 22:43:35.802019 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:35.802022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:35.802069 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:35.802145 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:35.803134 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:35.803138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:35.803141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:35.803153 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:35.803171 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:35.803179 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:35.803255 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:35.803380 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 22:43:35.803422 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:43:35.804338 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.799936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36909 (local address 127.26.189.193:37073)
0807 22:43:35.800558 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:35.800597 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:35.801821 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:35.802338 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:35.802346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:35.802445 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:35.802896 (+ 451us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:35.802897 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:35.802932 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:35.802996 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:35.803247 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:35.803255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:35.803261 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:43:35.803413 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 22:43:35.803637 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":115,"threads_started":1}
I20260807 22:43:35.805423 10635 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36909
I20260807 22:43:35.805629 10635 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:35.805994 10635 heartbeater.cc:511] Master 127.26.189.254:36909 requested a full tablet report, sending...
I20260807 22:43:35.806399 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 10437
I20260807 22:43:35.806506 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:35.806841 27383 external_mini_cluster.cc:1428] Setting key 5d792f9920b18a8aebba483f58c760f8
I20260807 22:43:35.807629 10343 ts_manager.cc:194] Registered new tserver with Master: 68d01f803cd94c9d9da9282e5c334ed0 (127.26.189.193:43175)
I20260807 22:43:35.808862 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36909
--builtin_ntp_servers=127.26.189.212:42505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:43:35.917423 10638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:35.917675 10638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:35.917730 10638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:35.921713 10638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:35.921761 10638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:35.921835 10638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:35.926613 10638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:35.927589 10638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:35.928918 10638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:35.938005 10643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.937999 10647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:35.937999 10645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:35.938112 10638 server_base.cc:1034] running on GCE node
I20260807 22:43:35.938658 10638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:35.939261 10638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:35.940435 10638 hybrid_clock.cc:648] HybridClock initialized: now 1786142615940420 us; error 26 us; skew 500 ppm
I20260807 22:43:35.942144 10638 webserver.cc:460] Webserver started at http://127.26.189.194:36509/ using document root <none> and password file <none>
I20260807 22:43:35.942708 10638 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:35.942775 10638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:35.942970 10638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:35.944908 10638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "93f32af18c024dbc97ce9622f45d356f"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "0a3c417b751ddb030eafa06fb0bcccab"
server_key_iv: "95e7dd17bf3032b48819ec2541da5526"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.945449 10638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "93f32af18c024dbc97ce9622f45d356f"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "0a3c417b751ddb030eafa06fb0bcccab"
server_key_iv: "95e7dd17bf3032b48819ec2541da5526"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.949085 10638 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 22:43:35.951577 10655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.952713 10638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:43:35.952817 10638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "93f32af18c024dbc97ce9622f45d356f"
format_stamp: "Formatted at 2026-08-07 22:43:35 on dist-test-slave-4tdj"
server_key: "0a3c417b751ddb030eafa06fb0bcccab"
server_key_iv: "95e7dd17bf3032b48819ec2541da5526"
server_key_version: "encryptionkey@0"
I20260807 22:43:35.952913 10638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:35.957219 10638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:35.957782 10638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:35.957981 10638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:35.958529 10638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:35.959414 10638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:35.959470 10638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.959518 10638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:35.959542 10638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:35.967963 10638 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:34639
I20260807 22:43:35.967990 10844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:34639 every 8 connection(s)
I20260807 22:43:35.968845 10638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:43:35.974442 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.970638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:57819 (local address 127.26.189.254:36909)
0807 22:43:35.970763 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:35.970767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:35.971769 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 22:43:35.973029 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:35.973032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:35.973069 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:35.973131 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:35.974054 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:35.974057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:35.974059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:35.974070 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:35.974090 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:35.974100 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:35.974169 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:35.974246 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 22:43:35.974282 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:43:35.975042 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 10638
I20260807 22:43:35.975155 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:35.975251 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:35.971001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36909 (local address 127.26.189.194:57819)
0807 22:43:35.971637 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:35.971675 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:35.972835 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:35.973273 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:35.973278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:35.973360 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:35.973822 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:35.973824 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:35.973861 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:35.973926 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:35.974152 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:35.974156 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:35.974159 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:43:35.974272 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:43:35.974506 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260807 22:43:35.975432 27383 external_mini_cluster.cc:1428] Setting key 20166b515f37f12924858a459a96e681
I20260807 22:43:35.976356 10845 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36909
I20260807 22:43:35.976549 10845 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:35.976967 10845 heartbeater.cc:511] Master 127.26.189.254:36909 requested a full tablet report, sending...
I20260807 22:43:35.977435 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36909
--builtin_ntp_servers=127.26.189.212:42505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 22:43:35.978024 10343 ts_manager.cc:194] Registered new tserver with Master: 93f32af18c024dbc97ce9622f45d356f (127.26.189.194:34639)
W20260807 22:43:36.086593 10851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:36.086841 10851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:36.086879 10851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:36.090870 10851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:36.090919 10851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:36.090970 10851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:36.095569 10851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:36909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:36.096460 10851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:36.097703 10851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:36.107040 10857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:36.107050 10860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:36.107017 10863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:36.107354 10851 server_base.cc:1034] running on GCE node
I20260807 22:43:36.107719 10851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:36.108305 10851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:36.109463 10851 hybrid_clock.cc:648] HybridClock initialized: now 1786142616109447 us; error 32 us; skew 500 ppm
I20260807 22:43:36.111183 10851 webserver.cc:460] Webserver started at http://127.26.189.195:36311/ using document root <none> and password file <none>
I20260807 22:43:36.111706 10851 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:36.111773 10851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:36.111974 10851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:36.113955 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "7c2b949dcbbe4431aec56bb1f0aa4496"
format_stamp: "Formatted at 2026-08-07 22:43:36 on dist-test-slave-4tdj"
server_key: "fd566965e83a5d56b7205872371ec649"
server_key_iv: "c6302f890cfbcb51eeecc17f983f176f"
server_key_version: "encryptionkey@0"
I20260807 22:43:36.114518 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "7c2b949dcbbe4431aec56bb1f0aa4496"
format_stamp: "Formatted at 2026-08-07 22:43:36 on dist-test-slave-4tdj"
server_key: "fd566965e83a5d56b7205872371ec649"
server_key_iv: "c6302f890cfbcb51eeecc17f983f176f"
server_key_version: "encryptionkey@0"
I20260807 22:43:36.117888 10851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260807 22:43:36.120299 10876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:36.121416 10851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:36.121522 10851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "7c2b949dcbbe4431aec56bb1f0aa4496"
format_stamp: "Formatted at 2026-08-07 22:43:36 on dist-test-slave-4tdj"
server_key: "fd566965e83a5d56b7205872371ec649"
server_key_iv: "c6302f890cfbcb51eeecc17f983f176f"
server_key_version: "encryptionkey@0"
I20260807 22:43:36.121618 10851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:36.125979 10851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:36.126616 10851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:36.126775 10851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:36.127288 10851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:36.128139 10851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:36.128186 10851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:36.128237 10851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:36.128261 10851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:36.136518 10851 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:44031
I20260807 22:43:36.136539 11041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:44031 every 8 connection(s)
I20260807 22:43:36.137311 10851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:36.142762 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:36.138915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:38675 (local address 127.26.189.254:36909)
0807 22:43:36.139072 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:36.139077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:36.140071 (+ 994us) server_negotiation.cc:407] Connection header received
0807 22:43:36.141306 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:36.141308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:36.141346 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:36.141407 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:36.142410 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:36.142413 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:36.142416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:43:36.142425 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:36.142440 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:43:36.142447 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:36.142514 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:36.142619 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 22:43:36.142656 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:43:36.143533 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:36.139308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36909 (local address 127.26.189.195:38675)
0807 22:43:36.139929 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:36.139968 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:36.141111 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:36.141544 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:36.141550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:36.141644 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:36.142151 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:43:36.142152 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:36.142190 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:43:36.142258 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:36.142506 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:36.142511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:36.142515 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:43:36.142644 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:43:36.142836 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":117,"threads_started":1}
I20260807 22:43:36.144163 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 10851
I20260807 22:43:36.144273 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:36.144608 27383 external_mini_cluster.cc:1428] Setting key d77c434fc210777c9d0a72581d34ec63
I20260807 22:43:36.144651 11042 heartbeater.cc:347] Connected to a master server at 127.26.189.254:36909
I20260807 22:43:36.144829 11042 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:36.145238 11042 heartbeater.cc:511] Master 127.26.189.254:36909 requested a full tablet report, sending...
I20260807 22:43:36.146270 10342 ts_manager.cc:194] Registered new tserver with Master: 7c2b949dcbbe4431aec56bb1f0aa4496 (127.26.189.195:44031)
I20260807 22:43:36.158115 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:43:36.160794 10404 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:43:36.159958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.26.189.254:36909)
0807 22:43:36.160072 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:36.160077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:36.160242 (+ 165us) server_negotiation.cc:407] Connection header received
0807 22:43:36.160348 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:36.160352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:36.160388 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:36.160449 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:36.160692 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57366: BlockingRecv error: recv got EOF from 127.0.0.1:57366 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:43:36.161073 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 10437
I20260807 22:43:36.173816 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 10638
I20260807 22:43:36.180154 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 10851
I20260807 22:43:36.188731 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 10295
2026-08-07T22:43:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (796 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:36 dist-test-slave-4tdj krb5kdc[11053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:36 dist-test-slave-4tdj krb5kdc[11053](info): set up 2 sockets
Aug 07 22:43:36 dist-test-slave-4tdj krb5kdc[11053](info): commencing operation
krb5kdc: starting...
W20260807 22:43:37.358222 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:37 dist-test-slave-4tdj krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:43:37.476362 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:44155
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:44155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:37.585433 11072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:37.585686 11072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:37.585739 11072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:37.589651 11072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:43:37.589704 11072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:37.589730 11072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:37.589749 11072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:43:37.589774 11072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:43:37.594914 11072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:44155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:44155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:37.595894 11072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:37.596866 11072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:37.604910 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.604923 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.604930 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:37.605049 11072 server_base.cc:1034] running on GCE node
I20260807 22:43:37.605777 11072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:37.606848 11072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:37.608009 11072 hybrid_clock.cc:648] HybridClock initialized: now 1786142617607992 us; error 29 us; skew 500 ppm
Aug 07 22:43:37 dist-test-slave-4tdj krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:37.614647 11072 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:43:37.615532 11072 webserver.cc:460] Webserver started at http://127.26.189.254:32801/ using document root <none> and password file <none>
I20260807 22:43:37.616063 11072 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:37.616127 11072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:37.616329 11072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:37.618366 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "b37f91a45c4a44538bf992ed87f3f8df"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-4tdj"
server_key: "b83e2e6753fe8386c9ed49f1b7ee8d16"
server_key_iv: "b0333c4ed05f97355c08866924518554"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.618921 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "b37f91a45c4a44538bf992ed87f3f8df"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-4tdj"
server_key: "b83e2e6753fe8386c9ed49f1b7ee8d16"
server_key_iv: "b0333c4ed05f97355c08866924518554"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.622498 11072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 22:43:37.624823 11091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.625978 11072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:37.626082 11072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "b37f91a45c4a44538bf992ed87f3f8df"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-4tdj"
server_key: "b83e2e6753fe8386c9ed49f1b7ee8d16"
server_key_iv: "b0333c4ed05f97355c08866924518554"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.626176 11072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:37.636253 11072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:37.638840 11072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:37.639002 11072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:37.645782 11072 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:44155
I20260807 22:43:37.646059 11163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:44155 every 8 connection(s)
I20260807 22:43:37.646731 11072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:43:37.649420 11165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:37.652026 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 11072
I20260807 22:43:37.652168 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:43:37.652472 27383 external_mini_cluster.cc:1428] Setting key 9214044d79d4a9ace3c763db9dc4a73c
I20260807 22:43:37.655315 11165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df: Bootstrap starting.
I20260807 22:43:37.657764 11165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:37.658658 11165 log.cc:824] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:37.660522 11165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df: No bootstrap required, opened a new log
Aug 07 22:43:37 dist-test-slave-4tdj krb5kdc[11053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:37.664283 11165 raft_consensus.cc:348] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f91a45c4a44538bf992ed87f3f8df" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44155 } }
I20260807 22:43:37.664554 11165 raft_consensus.cc:374] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:37.664624 11165 raft_consensus.cc:729] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37f91a45c4a44538bf992ed87f3f8df, State: Initialized, Role: FOLLOWER
I20260807 22:43:37.665091 11165 consensus_queue.cc:260] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f91a45c4a44538bf992ed87f3f8df" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44155 } }
I20260807 22:43:37.665251 11165 raft_consensus.cc:388] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:43:37.665342 11165 raft_consensus.cc:482] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:43:37.665445 11165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:37.666543 11165 raft_consensus.cc:504] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f91a45c4a44538bf992ed87f3f8df" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44155 } }
I20260807 22:43:37.666898 11165 leader_election.cc:302] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b37f91a45c4a44538bf992ed87f3f8df; no voters:
I20260807 22:43:37.667203 11165 leader_election.cc:288] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:43:37.667299 11174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:37.667498 11174 raft_consensus.cc:686] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [term 1 LEADER]: Becoming Leader. State: Replica: b37f91a45c4a44538bf992ed87f3f8df, State: Running, Role: LEADER
I20260807 22:43:37.667580 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.653803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.26.189.254:44155)
0807 22:43:37.654200 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:37.654221 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:37.654256 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:43:37.654924 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:37.654941 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:37.655258 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:37.655546 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:37.656403 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.657423 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.658617 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.659105 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.659260 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.661751 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:37.661757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:37.661773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:37.661804 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:37.664354 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.664840 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.664846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.664851 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.664988 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.665296 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.665299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.665301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.665803 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:37.665994 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:37.666269 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 22:43:37.666591 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":95,"threads_started":1}
I20260807 22:43:37.667872 11174 consensus_queue.cc:237] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f91a45c4a44538bf992ed87f3f8df" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44155 } }
I20260807 22:43:37.668378 11165 sys_catalog.cc:564] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:43:37.669492 11177 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [sys.catalog]: SysCatalogTable state changed. Reason: New leader b37f91a45c4a44538bf992ed87f3f8df. Latest consensus state: current_term: 1 leader_uuid: "b37f91a45c4a44538bf992ed87f3f8df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f91a45c4a44538bf992ed87f3f8df" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44155 } } }
I20260807 22:43:37.669605 11177 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:37.670013 11191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:43:37.669983 11176 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b37f91a45c4a44538bf992ed87f3f8df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f91a45c4a44538bf992ed87f3f8df" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 44155 } } }
I20260807 22:43:37.670127 11176 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df [sys.catalog]: This master's current role is: LEADER
I20260807 22:43:37.672879 11191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:43:37.677631 11191 catalog_manager.cc:1347] Generated new cluster ID: 99f96053f273476b9106c1c985f2d94b
I20260807 22:43:37.677695 11191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:43:37.698081 11191 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:43:37.699065 11191 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:43:37.706415 11191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df: Generated new TSK 0
I20260807 22:43:37.707065 11191 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:43:37.737591 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44155
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:37.848177 11211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:37.848410 11211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:37.848464 11211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:37.852471 11211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:37.852524 11211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:37.852592 11211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:43:37.857683 11211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:37.858624 11211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:37.859575 11211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:37.868462 11218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.868433 11222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:37.868448 11219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:37.868497 11211 server_base.cc:1034] running on GCE node
I20260807 22:43:37.869057 11211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:37.869675 11211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:37.870832 11211 hybrid_clock.cc:648] HybridClock initialized: now 1786142617870813 us; error 37 us; skew 500 ppm
Aug 07 22:43:37 dist-test-slave-4tdj krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:37.873502 11211 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:43:37.874313 11211 webserver.cc:460] Webserver started at http://127.26.189.193:35641/ using document root <none> and password file <none>
I20260807 22:43:37.874826 11211 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:37.874887 11211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:37.875077 11211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:37.876972 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "faa4c0a0883d42b2b125d48252bfc313"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-4tdj"
server_key: "483fb4565de67c2ecf7e20ccf0da3f8c"
server_key_iv: "d3c26ad026a371fec54cfd4c6eb06eb3"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.877496 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "faa4c0a0883d42b2b125d48252bfc313"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-4tdj"
server_key: "483fb4565de67c2ecf7e20ccf0da3f8c"
server_key_iv: "d3c26ad026a371fec54cfd4c6eb06eb3"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.880970 11211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:43:37.883247 11231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.884383 11211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:37.884498 11211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "faa4c0a0883d42b2b125d48252bfc313"
format_stamp: "Formatted at 2026-08-07 22:43:37 on dist-test-slave-4tdj"
server_key: "483fb4565de67c2ecf7e20ccf0da3f8c"
server_key_iv: "d3c26ad026a371fec54cfd4c6eb06eb3"
server_key_version: "encryptionkey@0"
I20260807 22:43:37.884590 11211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:37.896077 11211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:37.898715 11211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:37.898887 11211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:37.899432 11211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:37.900209 11211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:37.900256 11211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.900306 11211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:37.900336 11211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:37.908733 11211 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42891
I20260807 22:43:37.908758 11392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42891 every 8 connection(s)
I20260807 22:43:37.909545 11211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:43:37.913759 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 11211
I20260807 22:43:37.913858 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:43:37.914145 27383 external_mini_cluster.cc:1428] Setting key 62159e7c77cc5604e5540ae6daf015a6
Aug 07 22:43:37 dist-test-slave-4tdj krb5kdc[11053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142617, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:37.922659 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.911209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:57771 (local address 127.26.189.254:44155)
0807 22:43:37.911327 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:37.911330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:37.912083 (+ 753us) server_negotiation.cc:407] Connection header received
0807 22:43:37.913340 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:37.913343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:37.913391 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:37.913460 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:37.914647 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.915316 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.916424 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.916712 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.916800 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.919209 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:37.919213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:37.919215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:37.919244 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:37.920657 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.921160 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.921164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.921166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.921259 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:37.921553 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:37.921556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:37.921558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:37.921797 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:37.921903 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:37.922344 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 22:43:37.922475 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:43:37.923336 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:37.911525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44155 (local address 127.26.189.193:57771)
0807 22:43:37.911929 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:37.911970 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:37.913164 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:37.913580 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:37.913586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:37.913962 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:37.914494 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:37.914510 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.915445 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.915448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:37.916310 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:37.916320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:37.916855 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:37.916859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:37.917005 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:37.917476 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:37.917491 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:37.919043 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:37.920769 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:37.920774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:37.920785 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:37.921068 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:37.921340 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:37.921342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:37.921344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:37.921466 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:37.921886 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:37.921890 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:37.922103 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 22:43:37.922251 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:43:37.922532 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
I20260807 22:43:37.924499 11394 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44155
I20260807 22:43:37.924731 11394 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:43:37.925194 11394 heartbeater.cc:511] Master 127.26.189.254:44155 requested a full tablet report, sending...
I20260807 22:43:37.926649 11117 ts_manager.cc:194] Registered new tserver with Master: faa4c0a0883d42b2b125d48252bfc313 (127.26.189.193:42891)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:43:37.927977 11117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:57771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:43:37.940723 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44155
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:38.047956 11399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:38.048198 11399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:38.048238 11399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:38.052114 11399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:38.052161 11399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:38.052215 11399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:43:38.057231 11399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:38.058110 11399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:38.059067 11399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:38.068113 11408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:38.068150 11409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:38.068320 11399 server_base.cc:1034] running on GCE node
W20260807 22:43:38.068410 11413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:38.068768 11399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:38.069375 11399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:38.070526 11399 hybrid_clock.cc:648] HybridClock initialized: now 1786142618070522 us; error 45 us; skew 500 ppm
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.073029 11399 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:43:38.073808 11399 webserver.cc:460] Webserver started at http://127.26.189.194:38509/ using document root <none> and password file <none>
I20260807 22:43:38.074328 11399 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:38.074401 11399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:38.074612 11399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:38.076479 11399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "f1a9433d5f1e48d3965b3e69a548ee4c"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-4tdj"
server_key: "5651a4f2a0b580c772139f9c467a45ba"
server_key_iv: "80faaeda8893492011e818cb130dafc5"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.077001 11399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "f1a9433d5f1e48d3965b3e69a548ee4c"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-4tdj"
server_key: "5651a4f2a0b580c772139f9c467a45ba"
server_key_iv: "80faaeda8893492011e818cb130dafc5"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.080463 11399 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:43:38.082729 11422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.083820 11399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:43:38.083921 11399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "f1a9433d5f1e48d3965b3e69a548ee4c"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-4tdj"
server_key: "5651a4f2a0b580c772139f9c467a45ba"
server_key_iv: "80faaeda8893492011e818cb130dafc5"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.084004 11399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:38.093452 11399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:38.096001 11399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:38.096163 11399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:38.096681 11399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:38.097433 11399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:38.097481 11399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.097530 11399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:38.097559 11399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.105880 11399 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:36635
I20260807 22:43:38.105909 11590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:36635 every 8 connection(s)
I20260807 22:43:38.106714 11399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:38.116528 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 11399
I20260807 22:43:38.116647 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:43:38.116935 27383 external_mini_cluster.cc:1428] Setting key 7c7b8ed88a9faaed5839b5b66c506f90
I20260807 22:43:38.119998 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.108402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:40467 (local address 127.26.189.254:44155)
0807 22:43:38.108528 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.108531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.109304 (+ 773us) server_negotiation.cc:407] Connection header received
0807 22:43:38.110697 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.110700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.110762 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.110834 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.111975 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.112486 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.113575 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.113817 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.113906 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.116328 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.116331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.116333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.116358 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.117869 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.118392 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.118395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.118398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.118518 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.118880 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.118884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.118886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.119137 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.119247 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.119686 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.119824 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:43:38.120690 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.108716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44155 (local address 127.26.189.194:40467)
0807 22:43:38.109158 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.109200 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.110486 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.111011 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.111019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.111370 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:38.111826 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.111839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.112605 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.112607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.113454 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.113463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.113959 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.113963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.114128 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.114660 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:38.114675 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:38.116156 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:38.117977 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:38.117983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:38.117997 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:38.118273 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:38.118617 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:38.118620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:38.118622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:38.118768 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:38.119232 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:38.119237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:38.119448 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.119589 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.119829 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260807 22:43:38.121946 11591 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44155
I20260807 22:43:38.122164 11591 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:38.122658 11591 heartbeater.cc:511] Master 127.26.189.254:44155 requested a full tablet report, sending...
I20260807 22:43:38.123669 11117 ts_manager.cc:194] Registered new tserver with Master: f1a9433d5f1e48d3965b3e69a548ee4c (127.26.189.194:36635)
I20260807 22:43:38.124246 11117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:40467
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:43:38.144968 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44155
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:43:38.253547 11597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:43:38.253803 11597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:43:38.253859 11597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:43:38.257735 11597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:43:38.257791 11597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:43:38.257866 11597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:43:38.262887 11597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:44155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:43:38.263865 11597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:43:38.265008 11597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:43:38.274148 11607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:43:38.274345 11608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:38.274374 11597 server_base.cc:1034] running on GCE node
W20260807 22:43:38.274159 11610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:43:38.274828 11597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:43:38.275403 11597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:43:38.276547 11597 hybrid_clock.cc:648] HybridClock initialized: now 1786142618276522 us; error 42 us; skew 500 ppm
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:43:38.279081 11597 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:43:38.279860 11597 webserver.cc:460] Webserver started at http://127.26.189.195:39447/ using document root <none> and password file <none>
I20260807 22:43:38.280364 11597 fs_manager.cc:359] Metadata directory not provided
I20260807 22:43:38.280427 11597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:43:38.280634 11597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:43:38.282624 11597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "5708f8802c1d4a31b52e1e9611ab4c66"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-4tdj"
server_key: "c6ded784b8be71b56af0bb5223a81f01"
server_key_iv: "4d1ac739595d8c8d1a4b08b354687048"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.283138 11597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "5708f8802c1d4a31b52e1e9611ab4c66"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-4tdj"
server_key: "c6ded784b8be71b56af0bb5223a81f01"
server_key_iv: "4d1ac739595d8c8d1a4b08b354687048"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.286617 11597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:43:38.288970 11623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.290133 11597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:43:38.290238 11597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "5708f8802c1d4a31b52e1e9611ab4c66"
format_stamp: "Formatted at 2026-08-07 22:43:38 on dist-test-slave-4tdj"
server_key: "c6ded784b8be71b56af0bb5223a81f01"
server_key_iv: "4d1ac739595d8c8d1a4b08b354687048"
server_key_version: "encryptionkey@0"
I20260807 22:43:38.290359 11597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:43:38.306133 11597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:43:38.309058 11597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:43:38.309226 11597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:43:38.309778 11597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:43:38.310597 11597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:43:38.310653 11597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.310706 11597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:43:38.310732 11597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:43:38.318845 11597 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:42255
I20260807 22:43:38.318904 11811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:42255 every 8 connection(s)
I20260807 22:43:38.319679 11597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:43:38.320688 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 11597
I20260807 22:43:38.320806 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:43:38.321107 27383 external_mini_cluster.cc:1428] Setting key ecf4fdae92945b9f40da91780982352b
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142618, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:43:38.332639 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.321485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:44711 (local address 127.26.189.254:44155)
0807 22:43:38.321613 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.321617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.322572 (+ 955us) server_negotiation.cc:407] Connection header received
0807 22:43:38.324001 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.324004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.324059 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.324129 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.325235 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.325761 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.327143 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.327380 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.327514 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.329840 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.329843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.329846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.329873 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.331113 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.331529 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.331531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.331533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.331599 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.331838 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.331840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.331841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.331994 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.332061 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.332437 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.332525 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:43:38.333361 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.321920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:44155 (local address 127.26.189.195:44711)
0807 22:43:38.322426 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.322483 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.323809 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.324298 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.324305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.324645 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:43:38.325085 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.325097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.325875 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.325879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.327006 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.327018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.327542 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.327545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.327688 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.328199 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:43:38.328215 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:43:38.329691 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:43:38.331213 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:38.331218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:38.331228 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:38.331456 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:38.331660 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:43:38.331662 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:43:38.331663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:43:38.331776 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:43:38.332059 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:43:38.332063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:43:38.332264 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.332404 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.332637 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":120,"threads_started":1}
I20260807 22:43:38.334430 11813 heartbeater.cc:347] Connected to a master server at 127.26.189.254:44155
I20260807 22:43:38.334681 11813 heartbeater.cc:464] Registering TS with master...
I20260807 22:43:38.335143 11813 heartbeater.cc:511] Master 127.26.189.254:44155 requested a full tablet report, sending...
I20260807 22:43:38.336054 11116 ts_manager.cc:194] Registered new tserver with Master: 5708f8802c1d4a31b52e1e9611ab4c66 (127.26.189.195:42255)
I20260807 22:43:38.336576 11116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:44711
I20260807 22:43:38.346442 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:43:38.354525 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.348167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.26.189.254:44155)
0807 22:43:38.348264 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.348267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.348434 (+ 167us) server_negotiation.cc:407] Connection header received
0807 22:43:38.348529 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.348531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.348565 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.348626 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:43:38.349293 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.349780 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.350569 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.350767 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.350840 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.351971 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:43:38.351974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:43:38.351976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:43:38.351996 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:43:38.353287 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.353623 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.353625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.353626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.353691 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:43:38.353867 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:43:38.353869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:43:38.353870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:43:38.354054 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:43:38.354162 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.354320 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.354416 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 22:43:38.357124 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:43:38.360016 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:43:38.373416 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.368277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:36635 (local address 127.0.0.1:51718)
0807 22:43:38.368809 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.368822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.368906 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.369469 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.369472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.369491 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:38.369594 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.369601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.370652 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.370656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.371949 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.371959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.372862 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.372866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.372979 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.373019 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.373142 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.373262 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260807 22:43:38.373422 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.368440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.26.189.194:36635)
0807 22:43:38.369067 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.369078 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.369099 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:43:38.369170 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.369175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.369350 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.369491 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:38.369684 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.370536 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.372064 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.372740 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.372857 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.372981 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.373180 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.373273 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":115,"threads_started":1}
I20260807 22:43:38.373644 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.367680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:42255 (local address 127.0.0.1:43014)
0807 22:43:38.368466 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.368479 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.368554 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.369601 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.369604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.369622 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:38.369700 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.369706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.371221 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.371253 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.372415 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.372423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.373304 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.373307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.373387 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.373409 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.373456 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.373520 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":105,"threads_started":1}
I20260807 22:43:38.373867 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.367577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42891 (local address 127.0.0.1:42316)
0807 22:43:38.368041 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.368084 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.368202 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.369282 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.369288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.369312 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:38.369446 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.369453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.370961 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.370964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.372264 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.372276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.373450 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.373454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.373585 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.373608 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.373680 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.373747 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":45,"thread_start_us":65,"threads_started":1}
I20260807 22:43:38.374275 11832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.367723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.26.189.193:42891)
0807 22:43:38.368277 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.368292 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.368317 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:43:38.368910 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.368916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.369130 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.369277 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:38.369590 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.370817 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.372390 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.373155 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.373268 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.373383 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.374028 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.374132 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":106,"threads_started":1}
I20260807 22:43:38.374653 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.368791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.26.189.195:42255)
0807 22:43:38.369164 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.369172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.369190 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:43:38.369258 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.369263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.369445 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.369845 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:38.369987 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.371039 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.372523 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.373183 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.374199 (+ 1016us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.374365 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.374411 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.374494 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":112,"threads_started":1}
I20260807 22:43:38.376564 11488 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4ca1f9f20a4791acd976e314e88d44 (DEFAULT_TABLE table=test-table [id=75e63e2b34084fc9994bd5ea5f905b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:38.376853 11707 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4ca1f9f20a4791acd976e314e88d44 (DEFAULT_TABLE table=test-table [id=75e63e2b34084fc9994bd5ea5f905b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:38.376920 11294 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4ca1f9f20a4791acd976e314e88d44 (DEFAULT_TABLE table=test-table [id=75e63e2b34084fc9994bd5ea5f905b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:43:38.377713 11707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4ca1f9f20a4791acd976e314e88d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:38.377713 11488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4ca1f9f20a4791acd976e314e88d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:38.377779 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4ca1f9f20a4791acd976e314e88d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:43:38.383666 11837 tablet_bootstrap.cc:492] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Bootstrap starting.
I20260807 22:43:38.384976 11841 tablet_bootstrap.cc:492] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Bootstrap starting.
I20260807 22:43:38.386018 11837 tablet_bootstrap.cc:654] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:38.386591 11840 tablet_bootstrap.cc:492] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: Bootstrap starting.
I20260807 22:43:38.387403 11837 log.cc:824] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:38.387580 11841 tablet_bootstrap.cc:654] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:38.388427 11841 log.cc:824] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:38.388517 11840 tablet_bootstrap.cc:654] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: Neither blocks nor log segments found. Creating new log.
I20260807 22:43:38.389775 11840 log.cc:824] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: Log is configured to *not* fsync() on all Append() calls
I20260807 22:43:38.389858 11837 tablet_bootstrap.cc:492] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: No bootstrap required, opened a new log
I20260807 22:43:38.390025 11837 ts_tablet_manager.cc:1397] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 22:43:38.390410 11841 tablet_bootstrap.cc:492] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: No bootstrap required, opened a new log
I20260807 22:43:38.390588 11841 ts_tablet_manager.cc:1397] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:43:38.391703 11840 tablet_bootstrap.cc:492] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: No bootstrap required, opened a new log
I20260807 22:43:38.391873 11840 ts_tablet_manager.cc:1397] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:38.393038 11837 raft_consensus.cc:348] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.393294 11837 raft_consensus.cc:374] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:38.393353 11837 raft_consensus.cc:729] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5708f8802c1d4a31b52e1e9611ab4c66, State: Initialized, Role: FOLLOWER
I20260807 22:43:38.393810 11837 consensus_queue.cc:260] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.394171 11841 raft_consensus.cc:348] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.394408 11813 heartbeater.cc:503] Master 127.26.189.254:44155 was elected leader, sending a full tablet report...
I20260807 22:43:38.394461 11841 raft_consensus.cc:374] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:38.394512 11841 raft_consensus.cc:729] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a9433d5f1e48d3965b3e69a548ee4c, State: Initialized, Role: FOLLOWER
I20260807 22:43:38.394914 11841 consensus_queue.cc:260] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.395442 11840 raft_consensus.cc:348] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.395622 11837 ts_tablet_manager.cc:1428] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:43:38.395663 11840 raft_consensus.cc:374] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:43:38.395792 11840 raft_consensus.cc:729] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa4c0a0883d42b2b125d48252bfc313, State: Initialized, Role: FOLLOWER
I20260807 22:43:38.395812 11841 ts_tablet_manager.cc:1428] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:43:38.396139 11591 heartbeater.cc:503] Master 127.26.189.254:44155 was elected leader, sending a full tablet report...
I20260807 22:43:38.396229 11840 consensus_queue.cc:260] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.397037 11840 ts_tablet_manager.cc:1428] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:43:38.397042 11394 heartbeater.cc:503] Master 127.26.189.254:44155 was elected leader, sending a full tablet report...
W20260807 22:43:38.410569 11395 tablet.cc:2369] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:38.570613 11815 tablet.cc:2369] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:43:38.607758 11592 tablet.cc:2369] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:43:38.721640 11846 raft_consensus.cc:482] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:43:38.721851 11846 raft_consensus.cc:504] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.723027 11846 leader_election.cc:288] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faa4c0a0883d42b2b125d48252bfc313 (127.26.189.193:42891), f1a9433d5f1e48d3965b3e69a548ee4c (127.26.189.194:36635)
I20260807 22:43:38.726922 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.723303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:36635 (local address 127.26.189.195:57463)
0807 22:43:38.723410 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.723424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.723523 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.723847 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.723851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.723878 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:38.724005 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.724012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.724972 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.724976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.725827 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.725837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.726523 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.726527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.726633 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.726651 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.726712 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.726785 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":33}
I20260807 22:43:38.726922 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.723454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:57463 (local address 127.26.189.194:36635)
0807 22:43:38.723596 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.723601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.723616 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:43:38.723673 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.723676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.723728 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.723806 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:38.724182 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.724869 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.725951 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.726416 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.726523 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.726580 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.726749 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.726823 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:43:38.727205 11851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.723303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42891 (local address 127.26.189.195:56389)
0807 22:43:38.724021 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.724034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:43:38.724131 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:43:38.724360 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:43:38.724363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:43:38.724381 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:43:38.724517 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.724525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.725379 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.725383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.726177 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:43:38.726187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.726812 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.726816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:43:38.726910 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.726928 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:43:38.726992 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:43:38.727060 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":305,"thread_start_us":156,"threads_started":1}
I20260807 22:43:38.727622 11521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4ca1f9f20a4791acd976e314e88d44" candidate_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" is_pre_election: true
I20260807 22:43:38.727746 11832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.723369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:56389 (local address 127.26.189.193:42891)
0807 22:43:38.723496 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.723501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.724118 (+ 617us) server_negotiation.cc:407] Connection header received
0807 22:43:38.724214 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.724217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.724273 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.724342 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:43:38.724645 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.725266 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.726327 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.726704 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.726810 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.726858 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.727558 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.727636 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:43:38.727950 11521 raft_consensus.cc:2454] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5708f8802c1d4a31b52e1e9611ab4c66 in term 0.
I20260807 22:43:38.728299 11318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4ca1f9f20a4791acd976e314e88d44" candidate_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa4c0a0883d42b2b125d48252bfc313" is_pre_election: true
I20260807 22:43:38.728425 11625 leader_election.cc:302] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5708f8802c1d4a31b52e1e9611ab4c66, f1a9433d5f1e48d3965b3e69a548ee4c; no voters:
I20260807 22:43:38.728524 11318 raft_consensus.cc:2454] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5708f8802c1d4a31b52e1e9611ab4c66 in term 0.
I20260807 22:43:38.728655 11846 raft_consensus.cc:2781] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:43:38.728722 11846 raft_consensus.cc:482] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:43:38.728750 11846 raft_consensus.cc:3037] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:38.729786 11846 raft_consensus.cc:504] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.730126 11846 leader_election.cc:288] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [CANDIDATE]: Term 1 election: Requested vote from peers faa4c0a0883d42b2b125d48252bfc313 (127.26.189.193:42891), f1a9433d5f1e48d3965b3e69a548ee4c (127.26.189.194:36635)
I20260807 22:43:38.730439 11318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4ca1f9f20a4791acd976e314e88d44" candidate_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa4c0a0883d42b2b125d48252bfc313"
I20260807 22:43:38.730506 11521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4ca1f9f20a4791acd976e314e88d44" candidate_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c"
I20260807 22:43:38.730569 11318 raft_consensus.cc:3037] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:38.730613 11521 raft_consensus.cc:3037] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:43:38.731801 11318 raft_consensus.cc:2454] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5708f8802c1d4a31b52e1e9611ab4c66 in term 1.
I20260807 22:43:38.731801 11521 raft_consensus.cc:2454] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5708f8802c1d4a31b52e1e9611ab4c66 in term 1.
I20260807 22:43:38.732070 11625 leader_election.cc:302] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5708f8802c1d4a31b52e1e9611ab4c66, f1a9433d5f1e48d3965b3e69a548ee4c, faa4c0a0883d42b2b125d48252bfc313; no voters:
I20260807 22:43:38.732239 11846 raft_consensus.cc:2781] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:43:38.732476 11846 raft_consensus.cc:686] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 1 LEADER]: Becoming Leader. State: Replica: 5708f8802c1d4a31b52e1e9611ab4c66, State: Running, Role: LEADER
I20260807 22:43:38.732805 11846 consensus_queue.cc:237] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } }
I20260807 22:43:38.735754 11115 catalog_manager.cc:5495] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5708f8802c1d4a31b52e1e9611ab4c66 (127.26.189.195). New cstate: current_term: 1 leader_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5708f8802c1d4a31b52e1e9611ab4c66" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 42255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 } health_report { overall_health: UNKNOWN } } }
I20260807 22:43:38.820493 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.817362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.26.189.195:42255)
0807 22:43:38.817484 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:43:38.817488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:43:38.817502 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:43:38.817610 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:43:38.817612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:43:38.817672 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:43:38.817759 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:43:38.817937 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.818471 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.819294 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:43:38.819496 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:43:38.819565 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:43:38.819786 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:43:38.820187 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:43:38.820274 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:43:38.820319 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 22:43:38.820369 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:43:38.826140 11318 raft_consensus.cc:1261] T 4a4ca1f9f20a4791acd976e314e88d44 P faa4c0a0883d42b2b125d48252bfc313 [term 1 FOLLOWER]: Refusing update from remote peer 5708f8802c1d4a31b52e1e9611ab4c66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:38.826140 11521 raft_consensus.cc:1261] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 1 FOLLOWER]: Refusing update from remote peer 5708f8802c1d4a31b52e1e9611ab4c66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:43:38.826874 11852 consensus_queue.cc:1035] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1a9433d5f1e48d3965b3e69a548ee4c" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:38.827054 11846 consensus_queue.cc:1035] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faa4c0a0883d42b2b125d48252bfc313" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:43:38.832851 11858 mvcc.cc:205] Tried to move back new op lower bound from 7316040166705172480 to 7316040166331854848. Current Snapshot: MvccSnapshot[applied={T|T < 7316040166705172480}]
I20260807 22:43:38.843148 11115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:38.843291 11115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:43:38.845414 11115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b37f91a45c4a44538bf992ed87f3f8df: Sending DeleteTablet for 3 replicas of tablet 4a4ca1f9f20a4791acd976e314e88d44
I20260807 22:43:38.846083 11294 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4ca1f9f20a4791acd976e314e88d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:42316
I20260807 22:43:38.846159 11707 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4ca1f9f20a4791acd976e314e88d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:43014
I20260807 22:43:38.846658 11876 tablet_replica.cc:330] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: stopping tablet replica
I20260807 22:43:38.846621 11488 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4ca1f9f20a4791acd976e314e88d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:43:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51718
I20260807 22:43:38.846838 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 11211
I20260807 22:43:38.846936 11876 raft_consensus.cc:2229] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:43:38.849671 11876 raft_consensus.cc:2258] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:38.852803 11876 ts_tablet_manager.cc:1905] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:38.854039 11878 tablet_replica.cc:330] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: stopping tablet replica
I20260807 22:43:38.854362 11878 raft_consensus.cc:2229] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:43:38.854686 11878 raft_consensus.cc:2258] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:43:38.855482 11876 ts_tablet_manager.cc:1918] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:38.855568 11876 log.cc:1196] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/4a4ca1f9f20a4791acd976e314e88d44
I20260807 22:43:38.855835 11876 ts_tablet_manager.cc:1939] T 4a4ca1f9f20a4791acd976e314e88d44 P 5708f8802c1d4a31b52e1e9611ab4c66: Deleting consensus metadata
I20260807 22:43:38.856581 11878 ts_tablet_manager.cc:1905] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:43:38.856953 11093 catalog_manager.cc:4841] TS 5708f8802c1d4a31b52e1e9611ab4c66 (127.26.189.195:42255): tablet 4a4ca1f9f20a4791acd976e314e88d44 (table test-table [id=75e63e2b34084fc9994bd5ea5f905b19]) successfully deleted
I20260807 22:43:38.858801 11878 ts_tablet_manager.cc:1918] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:43:38.858899 11878 log.cc:1196] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/4a4ca1f9f20a4791acd976e314e88d44
I20260807 22:43:38.859191 11878 ts_tablet_manager.cc:1939] T 4a4ca1f9f20a4791acd976e314e88d44 P f1a9433d5f1e48d3965b3e69a548ee4c: Deleting consensus metadata
I20260807 22:43:38.860260 11093 catalog_manager.cc:4841] TS f1a9433d5f1e48d3965b3e69a548ee4c (127.26.189.194:36635): tablet 4a4ca1f9f20a4791acd976e314e88d44 (table test-table [id=75e63e2b34084fc9994bd5ea5f905b19]) successfully deleted
W20260807 22:43:38.862604 11095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:42891 (error 108)
I20260807 22:43:38.862872 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 11399
I20260807 22:43:38.874145 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:43:38.873810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42891 (local address 127.0.0.1:42330)
0807 22:43:38.873993 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 22:43:38.874058 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260807 22:43:38.874410 11095 catalog_manager.cc:4584] TS faa4c0a0883d42b2b125d48252bfc313 (127.26.189.193:42891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a4ca1f9f20a4791acd976e314e88d44: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42891: connect: Connection refused (error 111)
I20260807 22:43:38.879518 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 11597
I20260807 22:43:38.889961 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 11072
2026-08-07T22:43:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2698 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11883](info): set up 2 sockets
Aug 07 22:43:38 dist-test-slave-4tdj krb5kdc[11883](info): commencing operation
krb5kdc: starting...
W20260807 22:43:40.021668 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:43:40 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:43:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/rangeradmin_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/rangeradmin_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/rangerlookup_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/rangerlookup_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
I20260807 22:43:40.182277 27383 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskp7gIfB/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/postgres -l logfile start
2026-08-07 22:43:42.447 UTC [11924] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:43:42.447 UTC [11924] LOG: listening on IPv4 address "127.26.189.212", port 46585
2026-08-07 22:43:42.450 UTC [11924] LOG: listening on Unix socket "/tmp/.s.PGSQL.46585"
2026-08-07 22:43:42.476 UTC [11928] LOG: database system was shut down at 2026-08-07 22:43:41 UTC
2026-08-07 22:43:42.480 UTC [11924] LOG: database system is ready to accept connections
I20260807 22:43:43.528576 27383 mini_postgres.cc:96] Postgres bound to 46585
2026-08-07 22:43:43.537 UTC [11938] FATAL: database "slave" does not exist
127.26.189.212:46585 - accepting connections
I20260807 22:43:43.538064 27383 mini_ranger.cc:161] Starting Ranger...
I20260807 22:43:43.554203 27383 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:43:43.948119 27383 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:43:43.948206 27383 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-admin
2026-08-07 22:43:44,312 [I] DB FLAVOR :POSTGRES
2026-08-07 22:43:44,312 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:43:44,312 [I] Checking connection..
2026-08-07 22:43:44,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:43:44,566 [I] Checking connection passed.
2026-08-07 22:43:44,566 [I] --------- Verifying version history table ---------
2026-08-07 22:43:44,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:43:44,810 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:43:44,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:43:45,059 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:43:45,060 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:43:45,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:43:45.313 UTC [12032] WARNING: there is no transaction in progress
2026-08-07 22:43:45,321 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:43:45,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:43:45,565 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:43:45,565 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:43:45,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:43:45,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:43:46,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4tdj.c.gcp-upstream.internal','N') ;"
2026-08-07 22:43:46,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:43:46,550 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:43:46,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:43:46,799 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:43:46,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:43:47,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:43:47,310 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:43:47,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:43:48.197 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.212 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.227 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.240 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.254 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.273 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.323 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.329 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.341 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.350 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.360 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.372 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.381 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.386 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.392 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48.397 UTC [12188] WARNING: there is no transaction in progress
2026-08-07 22:43:48,964 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:43:48,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:43:49,197 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:43:49,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:43:49,445 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:43:49,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:43:49,693 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:43:49,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:43:49,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:43:50,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4tdj.c.gcp-upstream.internal';"
2026-08-07 22:43:50,448 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:43:50,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:43:50,694 [I] DB_PATCHES have already been applied
I20260807 22:43:50.700205 27383 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-admin:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskp7gIfB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:43:50.700271 27383 mini_ranger.cc:193] Using host: 127.26.189.212
I20260807 22:43:50.704510 27383 mini_ranger.cc:227] Ranger admin URL: http://127.26.189.212:33225
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:43:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:43:51.954754 27383 mini_ranger.cc:160] Time spent starting Ranger: real 8.417s user 0.003s sys 0.008s
Aug 07, 2026 10:44:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:44:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 10:44:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:44:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:44:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:44:12.656009 27383 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/rangerkms_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/rangerkms_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.189.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.189.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:44:12 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:44:12.758383 27383 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:44:12.768450 27383 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:44:13.517717 27383 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:44:13.517809 27383 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms
I20260807 22:44:13.517899 27383 mini_ranger_kms.cc:227] Using postgres at 127.26.189.212:46585
2026-08-07 22:44:13,833 [I] DB FLAVOR :POSTGRES
2026-08-07 22:44:13,834 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:44:13,834 [I] Checking connection
2026-08-07 22:44:13,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:44:14,073 [I] connection success
2026-08-07 22:44:14,074 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:44:14,074 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:44:14,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:44:14,315 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:44:14,316 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:44:14,316 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:44:14,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:46585/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:44:14,566 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:44:15.008349 27383 mini_ranger_kms.cc:310] Created kms service
I20260807 22:44:15.179574 27383 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:44:15.238528 27383 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:44:15 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:44:15.689795 27383 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:44:15.689951 27383 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms:/tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:44:15.689972 27383 mini_ranger_kms.cc:240] Using host: 127.26.189.212
I20260807 22:44:15.695248 27383 mini_ranger_kms.cc:276] Ranger KMS PID: 12455
I20260807 22:44:15.695369 27383 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.189.212:39851
Aug 07, 2026 10:44:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:44:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:44:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:44:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:44:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:44:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 22:44:17 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.189.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:18.155869 27383 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.398s user 0.012s sys 0.188s
I20260807 22:44:18.155999 27383 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:44:18.156100 27383 mini_ranger_kms.cc:401] 127.26.189.212:39851/kms/v1/keys
Aug 07 22:44:18 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.189.212@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
Aug 07 22:44:18 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.189.212@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
Aug 07 22:44:21 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:44:21.043387 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:46811
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:46811
--ranger_config_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:39851/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:21.164746 12540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:21.164989 12540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:21.165027 12540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:21.168932 12540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:44:21.168984 12540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:21.169003 12540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:21.169031 12540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:44:21.169067 12540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:44:21.174263 12540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.189.212:39851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:46811
--ranger_config_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:46811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:44:21.175216 12540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:21.176162 12540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:21.184755 12549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:21.184736 12546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:21.184808 12540 server_base.cc:1034] running on GCE node
W20260807 22:44:21.184779 12547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:21.185403 12540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:21.186486 12540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:21.187644 12540 hybrid_clock.cc:648] HybridClock initialized: now 1786142661187627 us; error 30 us; skew 500 ppm
Aug 07 22:44:21 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142661, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:21.194998 12540 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:44:21.195868 12540 webserver.cc:460] Webserver started at http://127.26.189.254:35747/ using document root <none> and password file <none>
I20260807 22:44:21.196341 12540 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:21.196391 12540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:21.196566 12540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:44:21 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142661, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:44:21.261699 12540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "3574273d0ad84ad1b7588c8d73f25eab"
format_stamp: "Formatted at 2026-08-07 22:44:21 on dist-test-slave-4tdj"
server_key: "a22c8c202c252adc5b5dc522d962cd81"
server_key_iv: "18f3683f3628bbecd958431e787cc4a7"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:21.262348 12540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "3574273d0ad84ad1b7588c8d73f25eab"
format_stamp: "Formatted at 2026-08-07 22:44:21 on dist-test-slave-4tdj"
server_key: "a22c8c202c252adc5b5dc522d962cd81"
server_key_iv: "18f3683f3628bbecd958431e787cc4a7"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:21.266543 12540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 22:44:21.283929 12557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:21.285559 12540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:44:21.285722 12540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "3574273d0ad84ad1b7588c8d73f25eab"
format_stamp: "Formatted at 2026-08-07 22:44:21 on dist-test-slave-4tdj"
server_key: "a22c8c202c252adc5b5dc522d962cd81"
server_key_iv: "18f3683f3628bbecd958431e787cc4a7"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:21.285868 12540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:21.310998 12540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:21.313421 12540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:21.313608 12540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:21.320770 12540 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:46811
I20260807 22:44:21.320785 12641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:46811 every 8 connection(s)
I20260807 22:44:21.321722 12540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:44:21.324596 12645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:21.330071 12645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab: Bootstrap starting.
I20260807 22:44:21.331631 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 12540
I20260807 22:44:21.331761 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:44:21.332077 27383 external_mini_cluster.cc:1428] Setting key 8806a60a060f00f67177ef08f348e7ab
I20260807 22:44:21.332854 12645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:21.333730 12645 log.cc:824] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:21.335803 12645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab: No bootstrap required, opened a new log
I20260807 22:44:21.339318 12645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3574273d0ad84ad1b7588c8d73f25eab" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46811 } }
I20260807 22:44:21.339564 12645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:21.339635 12645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3574273d0ad84ad1b7588c8d73f25eab, State: Initialized, Role: FOLLOWER
I20260807 22:44:21.340106 12645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3574273d0ad84ad1b7588c8d73f25eab" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46811 } }
I20260807 22:44:21.340241 12645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:44:21.340310 12645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 07 22:44:21 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:44:21.340416 12645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:21.341434 12645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3574273d0ad84ad1b7588c8d73f25eab" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46811 } }
I20260807 22:44:21.341773 12645 leader_election.cc:302] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3574273d0ad84ad1b7588c8d73f25eab; no voters:
I20260807 22:44:21.342105 12645 leader_election.cc:288] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:44:21.342170 12653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:21.342432 12653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [term 1 LEADER]: Becoming Leader. State: Replica: 3574273d0ad84ad1b7588c8d73f25eab, State: Running, Role: LEADER
I20260807 22:44:21.342737 12653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3574273d0ad84ad1b7588c8d73f25eab" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46811 } }
I20260807 22:44:21.343415 12645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:44:21.343905 12653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3574273d0ad84ad1b7588c8d73f25eab. Latest consensus state: current_term: 1 leader_uuid: "3574273d0ad84ad1b7588c8d73f25eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3574273d0ad84ad1b7588c8d73f25eab" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46811 } } }
I20260807 22:44:21.344018 12653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:21.344483 12655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3574273d0ad84ad1b7588c8d73f25eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3574273d0ad84ad1b7588c8d73f25eab" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 46811 } } }
I20260807 22:44:21.344635 12655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab [sys.catalog]: This master's current role is: LEADER
I20260807 22:44:21.344940 12658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:44:21.346217 12648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:21.333266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.26.189.254:46811)
0807 22:44:21.333719 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:21.333734 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:21.333769 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:44:21.334562 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:21.334581 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:21.334892 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:21.335198 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:21.336075 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.337041 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.337863 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:21.338364 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:21.338519 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:21.340961 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:21.340967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:21.340981 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:21.341008 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:21.342915 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:21.343414 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:21.343421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:21.343427 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:21.343575 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:21.343877 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:21.343881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:21.343883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:21.344416 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:21.344625 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:21.344925 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 22:44:21.345254 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":142,"threads_started":1}
I20260807 22:44:21.347811 12658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:44:21.348332 12645 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:44:21.355953 12680 mvcc.cc:205] Tried to move back new op lower bound from 7316040340903686144 to 7316040340860649472. Current Snapshot: MvccSnapshot[applied={T|T < 7316040340903686144}]
I20260807 22:44:21.357936 12658 catalog_manager.cc:1347] Generated new cluster ID: 28379302c5ea4a53b766d21dc2f2b0fa
I20260807 22:44:21.358065 12658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:44:21.373754 12658 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:44:21.375007 12658 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:44:21.387254 12658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab: Generated new TSK 0
I20260807 22:44:21.388025 12658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:44:22.099198 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:39851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46811
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:22.242494 12720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:22.242865 12720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:22.242996 12720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:22.248993 12720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:22.249068 12720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:22.249169 12720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:44:22.256711 12720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.189.212:39851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:44:22.258081 12720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:22.259509 12720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:22.271893 12731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:22.272310 12732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:22.272344 12739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:22.272619 12720 server_base.cc:1034] running on GCE node
I20260807 22:44:22.273067 12720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:22.273710 12720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:22.274894 12720 hybrid_clock.cc:648] HybridClock initialized: now 1786142662274864 us; error 45 us; skew 500 ppm
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:22.277994 12720 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:44:22.279148 12720 webserver.cc:460] Webserver started at http://127.26.189.193:42737/ using document root <none> and password file <none>
I20260807 22:44:22.279855 12720 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:22.279950 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:22.280215 12720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:44:22.313230 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "d7e19b45638849148d0d1f7727aa8fc4"
format_stamp: "Formatted at 2026-08-07 22:44:22 on dist-test-slave-4tdj"
server_key: "0b2d021c13e16b3221106e06b8494ed7"
server_key_iv: "f5a476540d127c14b4b5ee3db2b29c2b"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:22.313966 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "d7e19b45638849148d0d1f7727aa8fc4"
format_stamp: "Formatted at 2026-08-07 22:44:22 on dist-test-slave-4tdj"
server_key: "0b2d021c13e16b3221106e06b8494ed7"
server_key_iv: "f5a476540d127c14b4b5ee3db2b29c2b"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:22.326786 12720 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.007s sys 0.000s
I20260807 22:44:22.348596 12753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:22.350229 12720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:44:22.350384 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "d7e19b45638849148d0d1f7727aa8fc4"
format_stamp: "Formatted at 2026-08-07 22:44:22 on dist-test-slave-4tdj"
server_key: "0b2d021c13e16b3221106e06b8494ed7"
server_key_iv: "f5a476540d127c14b4b5ee3db2b29c2b"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:22.350524 12720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:22.374801 12720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:22.378201 12720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:22.378427 12720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:22.379168 12720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:22.380215 12720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:22.380275 12720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:22.380324 12720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:22.380396 12720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 22:44:22.391467 12720 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:34039
I20260807 22:44:22.391656 12932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:34039 every 8 connection(s)
I20260807 22:44:22.392453 12720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:44:22.394421 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 12720
I20260807 22:44:22.394569 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:44:22.394871 27383 external_mini_cluster.cc:1428] Setting key 2107283639cb41180b3a442c926364fd
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:44:22.410202 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.394239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:52481 (local address 127.26.189.254:46811)
0807 22:44:22.394578 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:22.394583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:22.397778 (+ 3195us) server_negotiation.cc:407] Connection header received
0807 22:44:22.397829 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:22.397832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:22.397887 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:22.397969 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:22.399547 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.400392 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.402057 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.402428 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.402544 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.405804 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:22.405808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:22.405811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:22.405841 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:22.407700 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:22.408360 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:22.408364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:22.408366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:22.408464 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:22.408888 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:22.408891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:22.408893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:22.409118 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:22.409189 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:22.409877 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 22:44:22.410012 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":100,"threads_started":1}
I20260807 22:44:22.411891 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.394730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:46811 (local address 127.26.189.193:52481)
0807 22:44:22.396414 (+ 1684us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:22.396469 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:22.397476 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:22.398313 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:22.398323 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:22.398690 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:22.399355 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.399375 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.400555 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.400559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.401915 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.401927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.402736 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.402741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.402950 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.403662 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:22.403683 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:22.405602 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:22.407847 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:22.407853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:22.407869 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:22.408219 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:22.408591 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:22.408594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:22.408596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:22.408777 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:22.409273 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:22.409279 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:22.409598 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 22:44:22.410408 (+ 810us) client_negotiation.cc:240] Negotiation successful
0807 22:44:22.410769 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1461,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:44:22.413345 12934 heartbeater.cc:347] Connected to a master server at 127.26.189.254:46811
I20260807 22:44:22.413619 12934 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:22.414191 12934 heartbeater.cc:511] Master 127.26.189.254:46811 requested a full tablet report, sending...
I20260807 22:44:22.416064 12581 ts_manager.cc:194] Registered new tserver with Master: d7e19b45638849148d0d1f7727aa8fc4 (127.26.189.193:34039)
I20260807 22:44:22.417595 12581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:52481
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:44:22.433189 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:39851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46811
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:44:22.586148 12944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:22.586580 12944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:22.586704 12944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:22.593004 12944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:22.593160 12944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:22.593304 12944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:44:22.600982 12944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.189.212:39851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:44:22.602330 12944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:22.603609 12944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:22.616077 12952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:22.616147 12955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:22.616394 12953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:22.617451 12944 server_base.cc:1034] running on GCE node
I20260807 22:44:22.617916 12944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:22.618752 12944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:22.619956 12944 hybrid_clock.cc:648] HybridClock initialized: now 1786142662619927 us; error 40 us; skew 500 ppm
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:22.623652 12944 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:44:22.624716 12944 webserver.cc:460] Webserver started at http://127.26.189.194:35161/ using document root <none> and password file <none>
I20260807 22:44:22.625414 12944 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:22.625499 12944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:22.625733 12944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:44:22.642597 12944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "c5c6447cb5fc4377b773c5dbe6277c2a"
format_stamp: "Formatted at 2026-08-07 22:44:22 on dist-test-slave-4tdj"
server_key: "24811555a17a55ddceb92fcb9b3fa888"
server_key_iv: "47157b29438be8f9516e9b53398daa83"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:22.643427 12944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "c5c6447cb5fc4377b773c5dbe6277c2a"
format_stamp: "Formatted at 2026-08-07 22:44:22 on dist-test-slave-4tdj"
server_key: "24811555a17a55ddceb92fcb9b3fa888"
server_key_iv: "47157b29438be8f9516e9b53398daa83"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:22.651777 12944 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.002s
I20260807 22:44:22.676275 12968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:22.677985 12944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:44:22.678128 12944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "c5c6447cb5fc4377b773c5dbe6277c2a"
format_stamp: "Formatted at 2026-08-07 22:44:22 on dist-test-slave-4tdj"
server_key: "24811555a17a55ddceb92fcb9b3fa888"
server_key_iv: "47157b29438be8f9516e9b53398daa83"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:22.678287 12944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:22.713526 12944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:22.717267 12944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:22.717514 12944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:22.718321 12944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:22.719408 12944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:22.719518 12944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:22.719622 12944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:22.719698 12944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:22.733790 12944 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:42551
I20260807 22:44:22.735054 12944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:44:22.735905 13142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:42551 every 8 connection(s)
I20260807 22:44:22.744491 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 12944
I20260807 22:44:22.744617 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:44:22.744916 27383 external_mini_cluster.cc:1428] Setting key 0eab3f7f8b507ff7e49305e1b11582a2
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:44:22.755591 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.739925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:47401 (local address 127.26.189.254:46811)
0807 22:44:22.740074 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:22.740079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:22.740640 (+ 561us) server_negotiation.cc:407] Connection header received
0807 22:44:22.741892 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:22.741896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:22.741966 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:22.742028 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:22.744337 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.745189 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.747803 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.748181 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.748262 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.751325 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:22.751329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:22.751332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:22.751361 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:22.753104 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:22.753726 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:22.753731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:22.753732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:22.753823 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:22.754227 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:22.754231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:22.754232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:22.754482 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:22.754569 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:22.755289 (+ 720us) server_negotiation.cc:299] Negotiation successful
0807 22:44:22.755425 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:44:22.757583 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:22.738774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:46811 (local address 127.26.189.194:47401)
0807 22:44:22.740465 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:22.740531 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:22.741623 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:22.742527 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:22.742536 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:22.742889 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:22.744076 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.744098 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.745581 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.745584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.747661 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:22.747674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:22.748470 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:22.748474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:22.748671 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:22.749329 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:22.749350 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:22.751134 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:22.753243 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:22.753249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:22.753266 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:22.753607 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:22.753924 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:22.753927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:22.753929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:22.754113 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:22.754639 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:22.754646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:22.754962 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 22:44:22.756100 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0807 22:44:22.756440 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1470,"thread_start_us":161,"threads_started":1}
I20260807 22:44:22.759131 13144 heartbeater.cc:347] Connected to a master server at 127.26.189.254:46811
I20260807 22:44:22.759368 13144 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:22.759933 13144 heartbeater.cc:511] Master 127.26.189.254:46811 requested a full tablet report, sending...
I20260807 22:44:22.761129 12581 ts_manager.cc:194] Registered new tserver with Master: c5c6447cb5fc4377b773c5dbe6277c2a (127.26.189.194:42551)
I20260807 22:44:22.761881 12581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:47401
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:44:22.809540 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:39851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46811
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:44:22 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
W20260807 22:44:23.028206 13150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:44:23.028625 13150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:44:23.028731 13150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:44:23.034520 13150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:44:23.034648 13150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:44:23.034770 13150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:44:23.045578 13150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.189.212:39851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:46811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:44:23.047259 13150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:44:23.048637 13150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:44:23.061575 13160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:23.062134 13157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:44:23.062463 13164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:44:23.067637 13150 server_base.cc:1034] running on GCE node
I20260807 22:44:23.068233 13150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:44:23.069027 13150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:44:23.072446 13150 hybrid_clock.cc:648] HybridClock initialized: now 1786142663072367 us; error 89 us; skew 500 ppm
Aug 07 22:44:23 dist-test-slave-4tdj krb5kdc[11883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142663, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:44:23.076310 13150 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:44:23.077606 13150 webserver.cc:460] Webserver started at http://127.26.189.195:42635/ using document root <none> and password file <none>
I20260807 22:44:23.078346 13150 fs_manager.cc:359] Metadata directory not provided
I20260807 22:44:23.078485 13150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:44:23.078779 13150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:44:23 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142663, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:44:23.096817 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "fe8f5818f2304152a356e2ee27fe57fd"
format_stamp: "Formatted at 2026-08-07 22:44:23 on dist-test-slave-4tdj"
server_key: "16edd58a12d25acbcd968f7f790b6af4"
server_key_iv: "1164c8411588e4382e128f88c6ef905e"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:23.097601 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "fe8f5818f2304152a356e2ee27fe57fd"
format_stamp: "Formatted at 2026-08-07 22:44:23 on dist-test-slave-4tdj"
server_key: "16edd58a12d25acbcd968f7f790b6af4"
server_key_iv: "1164c8411588e4382e128f88c6ef905e"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:23.104729 13150 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.000s
I20260807 22:44:23.128535 13177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:23.130188 13150 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:44:23.130335 13150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "fe8f5818f2304152a356e2ee27fe57fd"
format_stamp: "Formatted at 2026-08-07 22:44:23 on dist-test-slave-4tdj"
server_key: "16edd58a12d25acbcd968f7f790b6af4"
server_key_iv: "1164c8411588e4382e128f88c6ef905e"
server_key_version: "kuduclusterkey@0"
I20260807 22:44:23.130465 13150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:44:23.177791 13150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:44:23.181530 13150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:44:23.181785 13150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:44:23.182554 13150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:44:23.183946 13150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:44:23.184052 13150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:23.184151 13150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:44:23.184208 13150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:44:23.195487 13150 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:43923
I20260807 22:44:23.197161 13150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:44:23.199047 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 13150
I20260807 22:44:23.199167 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:44:23.199448 27383 external_mini_cluster.cc:1428] Setting key 3cc7ffa038f870e1e7bca555532140de
I20260807 22:44:23.200340 13350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:43923 every 8 connection(s)
Aug 07 22:44:23 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142663, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:44:23.217356 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.203755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:42199 (local address 127.26.189.254:46811)
0807 22:44:23.203904 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.203909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.204503 (+ 594us) server_negotiation.cc:407] Connection header received
0807 22:44:23.205704 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.205708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.205758 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.205803 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:23.207357 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.208154 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.209978 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.210335 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.210409 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.213389 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:23.213393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:23.213396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:23.213422 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:23.215056 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:23.215674 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:23.215678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:23.215680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:23.215765 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:23.216139 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:23.216142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:23.216144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:23.216347 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:23.216419 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.217079 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.217204 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:44:23.219122 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.202715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:46811 (local address 127.26.189.195:42199)
0807 22:44:23.204360 (+ 1645us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.204412 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.205463 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.206189 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.206198 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.206560 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:44:23.207184 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.207202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.208529 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.208533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.209844 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.209856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.210609 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.210613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.210801 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.211435 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:44:23.211455 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:44:23.213212 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:44:23.215190 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:23.215195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:23.215211 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:23.215561 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:23.215869 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:44:23.215872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:44:23.215874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:44:23.216039 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:44:23.216504 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:44:23.216510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:44:23.216814 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 22:44:23.217769 (+ 955us) client_negotiation.cc:240] Negotiation successful
0807 22:44:23.218090 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"thread_start_us":145,"threads_started":1}
I20260807 22:44:23.220582 13351 heartbeater.cc:347] Connected to a master server at 127.26.189.254:46811
I20260807 22:44:23.220831 13351 heartbeater.cc:464] Registering TS with master...
I20260807 22:44:23.221406 13351 heartbeater.cc:511] Master 127.26.189.254:46811 requested a full tablet report, sending...
I20260807 22:44:23.222579 12581 ts_manager.cc:194] Registered new tserver with Master: fe8f5818f2304152a356e2ee27fe57fd (127.26.189.195:43923)
I20260807 22:44:23.223286 12581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:42199
I20260807 22:44:23.225237 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:44:23.248322 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.228020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.26.189.254:46811)
0807 22:44:23.228143 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.228147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.228738 (+ 591us) server_negotiation.cc:407] Connection header received
0807 22:44:23.228929 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.228933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.228981 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.229033 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:44:23.235629 (+ 6596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.236426 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.242620 (+ 6194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.242961 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.243068 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.244914 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:44:23.244918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:44:23.244921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:44:23.244952 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:44:23.246558 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:23.247035 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:23.247038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:23.247040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:23.247123 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:44:23.247399 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:44:23.247402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:44:23.247404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:44:23.247617 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:44:23.247713 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.248050 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.248168 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:44:23.252111 12581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:44:23.260928 12581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 07 22:44:23 dist-test-slave-4tdj krb5kdc[11883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142662, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:44:23.306120 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.282540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.26.189.193:34039)
0807 22:44:23.286516 (+ 3976us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.286524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.296558 (+ 10034us) server_negotiation.cc:407] Connection header received
0807 22:44:23.296757 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.296764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.297020 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.297196 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:23.298853 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.300082 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.304196 (+ 4114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.305112 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.305231 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.305374 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.305824 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.305940 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3853,"thread_start_us":107,"threads_started":1}
I20260807 22:44:23.306650 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.281837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34039 (local address 127.0.0.1:59754)
0807 22:44:23.296377 (+ 14540us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.296437 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.296595 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.298508 (+ 1913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.298515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.298546 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:23.298700 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.298708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.300385 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.300389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.304045 (+ 3656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.304057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.305476 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.305480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.305610 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.305649 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:44:23.306366 (+ 717us) client_negotiation.cc:240] Negotiation successful
0807 22:44:23.306511 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14341,"thread_start_us":82,"threads_started":1}
I20260807 22:44:23.310369 12800 tablet_service.cc:1467] Processing CreateTablet for tablet f36d5840800b43ba85e9ba06fa72c8ef (DEFAULT_TABLE table=test-table [id=d47cea407ac74dd5a427b01091f22581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:44:23.311695 12800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36d5840800b43ba85e9ba06fa72c8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:23.315646 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.285278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:42551 (local address 127.0.0.1:50140)
0807 22:44:23.299097 (+ 13819us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.299113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.299205 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.302549 (+ 3344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.302553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.302574 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:23.302661 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.302667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.309777 (+ 7110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.309781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.310909 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.310922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.315232 (+ 4310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.315236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.315326 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.315351 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:44:23.315413 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 22:44:23.315488 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13751,"thread_start_us":102,"threads_started":1}
I20260807 22:44:23.317420 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.284049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:43923 (local address 127.0.0.1:60216)
0807 22:44:23.298487 (+ 14438us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.298502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.298608 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.300109 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.300112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.300130 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:23.300235 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.300241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.312356 (+ 12115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.312360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.313440 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.313451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.317022 (+ 3571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.317026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.317128 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.317151 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:44:23.317222 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:44:23.317294 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14367,"thread_start_us":94,"threads_started":1}
I20260807 22:44:23.318787 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.284484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.26.189.195:43923)
0807 22:44:23.299472 (+ 14988us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.299483 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.299508 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:44:23.299596 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.299602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.299808 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.299956 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:23.311085 (+ 11129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.312216 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.315912 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.316716 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.316837 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.316954 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.318544 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.318646 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14870,"thread_start_us":109,"threads_started":1}
I20260807 22:44:23.321723 13250 tablet_service.cc:1467] Processing CreateTablet for tablet f36d5840800b43ba85e9ba06fa72c8ef (DEFAULT_TABLE table=test-table [id=d47cea407ac74dd5a427b01091f22581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:44:23.322837 13250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36d5840800b43ba85e9ba06fa72c8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:23.331303 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.285667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.26.189.194:42551)
0807 22:44:23.301674 (+ 16007us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.301681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.301707 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:44:23.301781 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.301786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.301973 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.302119 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:23.302795 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.303956 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.314040 (+ 10084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.314893 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.315024 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.315160 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.331021 (+ 15861us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.331157 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15889,"thread_start_us":85,"threads_started":1}
I20260807 22:44:23.334899 13382 tablet_bootstrap.cc:492] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Bootstrap starting.
I20260807 22:44:23.334635 13046 tablet_service.cc:1467] Processing CreateTablet for tablet f36d5840800b43ba85e9ba06fa72c8ef (DEFAULT_TABLE table=test-table [id=d47cea407ac74dd5a427b01091f22581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:44:23.335943 13046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36d5840800b43ba85e9ba06fa72c8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:44:23.337785 13382 tablet_bootstrap.cc:654] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:23.338712 13382 log.cc:824] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:23.339569 13380 tablet_bootstrap.cc:492] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: Bootstrap starting.
I20260807 22:44:23.343060 13380 tablet_bootstrap.cc:654] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:23.344096 13380 log.cc:824] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:23.346426 13382 tablet_bootstrap.cc:492] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: No bootstrap required, opened a new log
I20260807 22:44:23.346712 13382 ts_tablet_manager.cc:1397] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260807 22:44:23.350697 13382 raft_consensus.cc:348] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.352538 13380 tablet_bootstrap.cc:492] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: No bootstrap required, opened a new log
I20260807 22:44:23.352671 13382 raft_consensus.cc:374] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:23.352743 13382 raft_consensus.cc:729] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe8f5818f2304152a356e2ee27fe57fd, State: Initialized, Role: FOLLOWER
I20260807 22:44:23.352913 13388 tablet_bootstrap.cc:492] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: Bootstrap starting.
I20260807 22:44:23.353394 13380 ts_tablet_manager.cc:1397] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260807 22:44:23.355132 13382 consensus_queue.cc:260] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.355329 13388 tablet_bootstrap.cc:654] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: Neither blocks nor log segments found. Creating new log.
I20260807 22:44:23.356220 13388 log.cc:824] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:44:23.356424 13382 ts_tablet_manager.cc:1428] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260807 22:44:23.357089 13351 heartbeater.cc:503] Master 127.26.189.254:46811 was elected leader, sending a full tablet report...
I20260807 22:44:23.359818 13388 tablet_bootstrap.cc:492] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: No bootstrap required, opened a new log
I20260807 22:44:23.360008 13388 ts_tablet_manager.cc:1397] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 22:44:23.359835 13380 raft_consensus.cc:348] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.360096 13380 raft_consensus.cc:374] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:23.360152 13380 raft_consensus.cc:729] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e19b45638849148d0d1f7727aa8fc4, State: Initialized, Role: FOLLOWER
I20260807 22:44:23.360613 13380 consensus_queue.cc:260] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.361598 13380 ts_tablet_manager.cc:1428] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260807 22:44:23.361901 12934 heartbeater.cc:503] Master 127.26.189.254:46811 was elected leader, sending a full tablet report...
I20260807 22:44:23.364284 13388 raft_consensus.cc:348] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.364533 13388 raft_consensus.cc:374] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:44:23.364585 13388 raft_consensus.cc:729] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c6447cb5fc4377b773c5dbe6277c2a, State: Initialized, Role: FOLLOWER
I20260807 22:44:23.365284 13388 consensus_queue.cc:260] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.366343 13388 ts_tablet_manager.cc:1428] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:44:23.366725 13144 heartbeater.cc:503] Master 127.26.189.254:46811 was elected leader, sending a full tablet report...
I20260807 22:44:23.367240 13391 raft_consensus.cc:482] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:44:23.367357 13391 raft_consensus.cc:504] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.369277 13391 leader_election.cc:288] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe8f5818f2304152a356e2ee27fe57fd (127.26.189.195:43923), c5c6447cb5fc4377b773c5dbe6277c2a (127.26.189.194:42551)
I20260807 22:44:23.373270 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.368683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:56375 (local address 127.26.189.195:43923)
0807 22:44:23.368798 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.368803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.369001 (+ 198us) server_negotiation.cc:407] Connection header received
0807 22:44:23.369212 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.369215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.369262 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.369310 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:23.369751 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.370448 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.371927 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.372549 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.372641 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.372691 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.373058 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.373136 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:44:23.373844 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.368575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:43923 (local address 127.26.189.193:56375)
0807 22:44:23.368924 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.368937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.369085 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.369455 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.369457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.369475 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:23.369634 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.369641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.370621 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.370624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.371796 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.371807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.372810 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.372814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.372918 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.372942 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:44:23.373615 (+ 673us) client_negotiation.cc:240] Negotiation successful
0807 22:44:23.373692 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":99,"threads_started":1}
I20260807 22:44:23.374544 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36d5840800b43ba85e9ba06fa72c8ef" candidate_uuid: "d7e19b45638849148d0d1f7727aa8fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe8f5818f2304152a356e2ee27fe57fd" is_pre_election: true
I20260807 22:44:23.374823 13256 raft_consensus.cc:2454] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e19b45638849148d0d1f7727aa8fc4 in term 0.
I20260807 22:44:23.375336 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.369444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:43119 (local address 127.26.189.194:42551)
0807 22:44:23.369908 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.369913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.369925 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:44:23.369964 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.369966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.370017 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.370080 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:44:23.371529 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.372190 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.374038 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.374670 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.374758 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.374801 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.375154 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.375229 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390}
I20260807 22:44:23.375377 12756 leader_election.cc:302] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e19b45638849148d0d1f7727aa8fc4, fe8f5818f2304152a356e2ee27fe57fd; no voters:
I20260807 22:44:23.375654 13391 raft_consensus.cc:2781] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:44:23.375725 13391 raft_consensus.cc:482] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:44:23.375782 13391 raft_consensus.cc:3037] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:23.375902 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.369044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:42551 (local address 127.26.189.193:43119)
0807 22:44:23.369524 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.369537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:44:23.369610 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:44:23.371271 (+ 1661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:44:23.371275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:44:23.371293 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:44:23.371408 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.371413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.372747 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.372750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.373877 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:44:23.373890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.374897 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.374900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:44:23.375010 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.375031 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:44:23.375646 (+ 615us) client_negotiation.cc:240] Negotiation successful
0807 22:44:23.375718 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":75,"threads_started":1}
I20260807 22:44:23.376416 13053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36d5840800b43ba85e9ba06fa72c8ef" candidate_uuid: "d7e19b45638849148d0d1f7727aa8fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" is_pre_election: true
I20260807 22:44:23.376668 13053 raft_consensus.cc:2454] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e19b45638849148d0d1f7727aa8fc4 in term 0.
I20260807 22:44:23.377041 13391 raft_consensus.cc:504] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.377727 13256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36d5840800b43ba85e9ba06fa72c8ef" candidate_uuid: "d7e19b45638849148d0d1f7727aa8fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe8f5818f2304152a356e2ee27fe57fd"
I20260807 22:44:23.377837 13256 raft_consensus.cc:3037] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:23.379102 13256 raft_consensus.cc:2454] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e19b45638849148d0d1f7727aa8fc4 in term 1.
I20260807 22:44:23.379487 12756 leader_election.cc:302] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e19b45638849148d0d1f7727aa8fc4, fe8f5818f2304152a356e2ee27fe57fd; no voters:
I20260807 22:44:23.379734 13393 raft_consensus.cc:2781] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:44:23.379942 13393 raft_consensus.cc:686] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [term 1 LEADER]: Becoming Leader. State: Replica: d7e19b45638849148d0d1f7727aa8fc4, State: Running, Role: LEADER
I20260807 22:44:23.380266 13393 consensus_queue.cc:237] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } }
I20260807 22:44:23.383482 12581 catalog_manager.cc:5495] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e19b45638849148d0d1f7727aa8fc4 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "d7e19b45638849148d0d1f7727aa8fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e19b45638849148d0d1f7727aa8fc4" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 } health_report { overall_health: UNKNOWN } } }
I20260807 22:44:23.385340 13053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36d5840800b43ba85e9ba06fa72c8ef" candidate_uuid: "d7e19b45638849148d0d1f7727aa8fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a"
I20260807 22:44:23.385460 13053 raft_consensus.cc:3037] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:44:23.386821 13053 raft_consensus.cc:2454] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e19b45638849148d0d1f7727aa8fc4 in term 1.
I20260807 22:44:23.387156 13391 leader_election.cc:288] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [CANDIDATE]: Term 1 election: Requested vote from peers fe8f5818f2304152a356e2ee27fe57fd (127.26.189.195:43923), c5c6447cb5fc4377b773c5dbe6277c2a (127.26.189.194:42551)
W20260807 22:44:23.394162 12936 tablet.cc:2369] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:44:23.409021 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.403410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.26.189.193:34039)
0807 22:44:23.403546 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:44:23.403552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:44:23.403566 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:44:23.403636 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:44:23.403639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:44:23.403699 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:44:23.403805 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:44:23.404143 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.404910 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.406279 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:44:23.406663 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:44:23.406764 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:44:23.407522 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:44:23.408033 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:44:23.408145 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:44:23.408818 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 22:44:23.408883 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:44:23.416167 13256 raft_consensus.cc:1261] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 1 FOLLOWER]: Refusing update from remote peer d7e19b45638849148d0d1f7727aa8fc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:44:23.416167 13053 raft_consensus.cc:1261] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a [term 1 FOLLOWER]: Refusing update from remote peer d7e19b45638849148d0d1f7727aa8fc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:44:23.416993 13391 consensus_queue.cc:1035] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c6447cb5fc4377b773c5dbe6277c2a" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:44:23.417191 13393 consensus_queue.cc:1035] T f36d5840800b43ba85e9ba06fa72c8ef P d7e19b45638849148d0d1f7727aa8fc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe8f5818f2304152a356e2ee27fe57fd" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 43923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:44:23.432374 13407 mvcc.cc:205] Tried to move back new op lower bound from 7316040349344686080 to 7316040349208633344. Current Snapshot: MvccSnapshot[applied={T|T < 7316040349344686080}]
I20260807 22:44:23.435819 13409 mvcc.cc:205] Tried to move back new op lower bound from 7316040349344686080 to 7316040349208633344. Current Snapshot: MvccSnapshot[applied={T|T < 7316040349344686080}]
I20260807 22:44:23.445469 13408 mvcc.cc:205] Tried to move back new op lower bound from 7316040349344686080 to 7316040349208633344. Current Snapshot: MvccSnapshot[applied={T|T < 7316040349344686080}]
I20260807 22:44:23.447754 12580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:44:23.447940 12580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:44:23.450716 12580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3574273d0ad84ad1b7588c8d73f25eab: Sending DeleteTablet for 3 replicas of tablet f36d5840800b43ba85e9ba06fa72c8ef
I20260807 22:44:23.451442 12800 tablet_service.cc:1514] Processing DeleteTablet for tablet f36d5840800b43ba85e9ba06fa72c8ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:44:23 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:59754
I20260807 22:44:23.452476 13250 tablet_service.cc:1514] Processing DeleteTablet for tablet f36d5840800b43ba85e9ba06fa72c8ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:44:23 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60216
I20260807 22:44:23.453241 13046 tablet_service.cc:1514] Processing DeleteTablet for tablet f36d5840800b43ba85e9ba06fa72c8ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:44:23 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:50140
I20260807 22:44:23.453430 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 12720
I20260807 22:44:23.458482 13420 tablet_replica.cc:330] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: stopping tablet replica
I20260807 22:44:23.458774 13420 raft_consensus.cc:2229] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:44:23.459062 13420 raft_consensus.cc:2258] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:44:23.460625 13420 ts_tablet_manager.cc:1905] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:44:23.463155 13420 ts_tablet_manager.cc:1918] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:44:23.463248 13420 log.cc:1196] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/f36d5840800b43ba85e9ba06fa72c8ef
I20260807 22:44:23.464311 13420 ts_tablet_manager.cc:1939] T f36d5840800b43ba85e9ba06fa72c8ef P fe8f5818f2304152a356e2ee27fe57fd: Deleting consensus metadata
I20260807 22:44:23.465672 12560 catalog_manager.cc:4841] TS fe8f5818f2304152a356e2ee27fe57fd (127.26.189.195:43923): tablet f36d5840800b43ba85e9ba06fa72c8ef (table test-table [id=d47cea407ac74dd5a427b01091f22581]) successfully deleted
W20260807 22:44:23.469466 12561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:34039 (error 108)
I20260807 22:44:23.470469 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.470176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34039 (local address 127.0.0.1:59774)
0807 22:44:23.470312 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.470380 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 22:44:23.470613 12561 catalog_manager.cc:4584] TS d7e19b45638849148d0d1f7727aa8fc4 (127.26.189.193:34039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f36d5840800b43ba85e9ba06fa72c8ef: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34039: connect: Connection refused (error 111)
I20260807 22:44:23.470944 13421 tablet_replica.cc:330] T f36d5840800b43ba85e9ba06fa72c8ef P c5c6447cb5fc4377b773c5dbe6277c2a: stopping tablet replica
I20260807 22:44:23.470958 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 12944
I20260807 22:44:23.480981 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 13150
I20260807 22:44:23.486702 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:44:23.485942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:42551 (local address 127.0.0.1:50156)
0807 22:44:23.486535 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 22:44:23.486603 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.194:42551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487}
I20260807 22:44:23.490485 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 12540
I20260807 22:44:23.503394 27383 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:44:23.847600 27383 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:44:23.847736 27383 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:44:24.191529 27383 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:44:24.191 UTC [11924] LOG: received smart shutdown request
2026-08-07 22:44:24.195 UTC [11924] LOG: background worker "logical replication launcher" (PID 11935) exited with exit code 1
2026-08-07 22:44:24.195 UTC [11929] LOG: shutting down
2026-08-07 22:44:24.342 UTC [11924] LOG: database system is shut down
2026-08-07T22:44:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45545 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:44:24 dist-test-slave-4tdj krb5kdc[13438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:44:24 dist-test-slave-4tdj krb5kdc[13438](info): set up 2 sockets
Aug 07 22:44:24 dist-test-slave-4tdj krb5kdc[13438](info): commencing operation
krb5kdc: starting...
W20260807 22:44:25.574536 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:44:25 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:44:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/rangeradmin_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/rangeradmin_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/rangerlookup_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/rangerlookup_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
I20260807 22:44:25.738231 27383 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskp7gIfB/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/postgres -l logfile start
2026-08-07 22:44:27.979 UTC [13484] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:44:27.979 UTC [13484] LOG: listening on IPv4 address "127.26.189.212", port 50675
2026-08-07 22:44:27.983 UTC [13484] LOG: listening on Unix socket "/tmp/.s.PGSQL.50675"
2026-08-07 22:44:28.010 UTC [13489] LOG: database system was shut down at 2026-08-07 22:44:26 UTC
2026-08-07 22:44:28.014 UTC [13484] LOG: database system is ready to accept connections
I20260807 22:44:29.050486 27383 mini_postgres.cc:96] Postgres bound to 50675
2026-08-07 22:44:29.058 UTC [13502] FATAL: database "slave" does not exist
127.26.189.212:50675 - accepting connections
I20260807 22:44:29.058905 27383 mini_ranger.cc:161] Starting Ranger...
I20260807 22:44:29.074108 27383 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:44:29.473594 27383 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:44:29.473680 27383 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-admin
2026-08-07 22:44:29,747 [I] DB FLAVOR :POSTGRES
2026-08-07 22:44:29,747 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:44:29,747 [I] Checking connection..
2026-08-07 22:44:29,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:44:30,004 [I] Checking connection passed.
2026-08-07 22:44:30,004 [I] --------- Verifying version history table ---------
2026-08-07 22:44:30,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:44:30,237 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:44:30,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:44:30,487 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:44:30,487 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:44:30,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:44:30.754 UTC [13599] WARNING: there is no transaction in progress
2026-08-07 22:44:30,761 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:44:30,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:44:31,000 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:44:31,000 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:44:31,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:44:31,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:44:31,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4tdj.c.gcp-upstream.internal','N') ;"
2026-08-07 22:44:31,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:44:31,989 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:44:31,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:44:32,222 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:44:32,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:44:32,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:44:32,725 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:44:32,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:44:33.639 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.654 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.668 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.682 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.699 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.718 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.781 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.787 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.800 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.809 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.819 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.831 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.840 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.846 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.852 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:33.857 UTC [13775] WARNING: there is no transaction in progress
2026-08-07 22:44:34,455 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:44:34,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:44:34,925 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:44:34,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:44:35,182 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:44:35,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:44:35,426 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:44:35,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:44:35,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:44:35,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4tdj.c.gcp-upstream.internal';"
2026-08-07 22:44:36,179 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:44:36,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:44:36,428 [I] DB_PATCHES have already been applied
I20260807 22:44:36.433722 27383 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-admin:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskp7gIfB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:44:36.433789 27383 mini_ranger.cc:193] Using host: 127.26.189.212
I20260807 22:44:36.437988 27383 mini_ranger.cc:227] Ranger admin URL: http://127.26.189.212:56177
Aug 07, 2026 10:44:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:44:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:44:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:44:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:44:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:44:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:44:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:44:37.681380 27383 mini_ranger.cc:160] Time spent starting Ranger: real 8.622s user 0.000s sys 0.011s
Aug 07, 2026 10:44:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:44:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 10:44:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:44:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:44:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:44:59.058861 27383 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.189.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.189.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/rangerkms_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.189.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/rangerkms_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.189.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.189.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.189.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/HTTP_127.26.189.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:44:59 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142699, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:44:59.136786 27383 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:44:59.145623 27383 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:44:59.856016 27383 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:44:59.856091 27383 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms
I20260807 22:44:59.856110 27383 mini_ranger_kms.cc:227] Using postgres at 127.26.189.212:50675
2026-08-07 22:45:00,020 [I] DB FLAVOR :POSTGRES
2026-08-07 22:45:00,021 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:45:00,021 [I] Checking connection
2026-08-07 22:45:00,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:45:00,254 [I] connection success
2026-08-07 22:45:00,254 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:45:00,254 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:45:00,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:45:00,484 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:45:00,485 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:45:00,485 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:45:00,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.189.212:50675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:45:00,740 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:45:01.226987 27383 mini_ranger_kms.cc:310] Created kms service
I20260807 22:45:01.413662 27383 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:45:01.458730 27383 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:45:01 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142699, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:01.863955 27383 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:45:01.864229 27383 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms:/tmp/dist-test-taskp7gIfB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskp7gIfB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:45:01.864259 27383 mini_ranger_kms.cc:240] Using host: 127.26.189.212
I20260807 22:45:01.869436 27383 mini_ranger_kms.cc:276] Ranger KMS PID: 14067
I20260807 22:45:01.869505 27383 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.189.212:35993
Aug 07, 2026 10:45:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:45:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:45:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:45:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:45:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:45:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 22:45:04 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142704, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.189.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:04.321347 27383 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.185s user 0.002s sys 0.146s
I20260807 22:45:04.321477 27383 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:45:04.321563 27383 mini_ranger_kms.cc:401] 127.26.189.212:35993/kms/v1/keys
Aug 07 22:45:04 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142704, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.189.212@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
Aug 07 22:45:04 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142704, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.189.212@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
Aug 07 22:45:07 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:45:07.098412 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39777
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39777
--ranger_config_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:07.211208 14148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:07.211468 14148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:07.211508 14148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:07.215487 14148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:07.215538 14148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:07.215555 14148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:07.215569 14148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:07.215582 14148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:07.220911 14148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.189.212:35993/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39777
--ranger_config_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:07.221870 14148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:07.222860 14148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:07.232064 14160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:07.232054 14161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:07.232548 14164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:07.232602 14148 server_base.cc:1034] running on GCE node
I20260807 22:45:07.233148 14148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:07.234246 14148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:07.235442 14148 hybrid_clock.cc:648] HybridClock initialized: now 1786142707235428 us; error 39 us; skew 500 ppm
Aug 07 22:45:07 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142707, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:07.241011 14148 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:07.241981 14148 webserver.cc:460] Webserver started at http://127.26.189.254:33099/ using document root <none> and password file <none>
I20260807 22:45:07.242532 14148 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:07.242595 14148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:07.242806 14148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:07 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142707, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:07.304105 14148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "cfa3616bab5040588a61480af614b866"
format_stamp: "Formatted at 2026-08-07 22:45:07 on dist-test-slave-4tdj"
server_key: "235a932d1c5546093b050ddfdfbb8cff"
server_key_iv: "044e96f5549f29981b3eaffb31377229"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:07.304713 14148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "cfa3616bab5040588a61480af614b866"
format_stamp: "Formatted at 2026-08-07 22:45:07 on dist-test-slave-4tdj"
server_key: "235a932d1c5546093b050ddfdfbb8cff"
server_key_iv: "044e96f5549f29981b3eaffb31377229"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:07.308511 14148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:45:07.324019 14174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:07.325587 14148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:45:07.325744 14148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "cfa3616bab5040588a61480af614b866"
format_stamp: "Formatted at 2026-08-07 22:45:07 on dist-test-slave-4tdj"
server_key: "235a932d1c5546093b050ddfdfbb8cff"
server_key_iv: "044e96f5549f29981b3eaffb31377229"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:07.325881 14148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:07.350682 14148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:07.353781 14148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:07.353973 14148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:07.362403 14263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:39777 every 8 connection(s)
I20260807 22:45:07.362412 14148 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:39777
I20260807 22:45:07.363586 14148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:07.365152 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 14148
I20260807 22:45:07.365298 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:45:07.365597 27383 external_mini_cluster.cc:1428] Setting key 0970b907367f6c23112f27f5f591a6d5
I20260807 22:45:07.366657 14264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:07.373153 14264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Bootstrap starting.
I20260807 22:45:07.375689 14264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Neither blocks nor log segments found. Creating new log.
Aug 07 22:45:07 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:07.376487 14264 log.cc:824] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:07.378509 14264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: No bootstrap required, opened a new log
I20260807 22:45:07.381822 14264 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:07.382037 14264 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:07.382130 14264 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa3616bab5040588a61480af614b866, State: Initialized, Role: FOLLOWER
I20260807 22:45:07.382550 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:07.367211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.26.189.254:39777)
0807 22:45:07.367678 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:07.367696 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:07.367738 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:45:07.368599 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:07.368621 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:07.369020 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:07.369352 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:07.370258 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:07.371821 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:07.372981 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:07.373657 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:07.373812 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:07.376748 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:07.376755 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:07.376772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:07.376803 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:07.379399 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:07.379894 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:07.379900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:07.379906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:07.380044 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:07.380336 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:07.380339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:07.380341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:07.380839 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:07.381032 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:07.381297 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 22:45:07.381578 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":146,"threads_started":1}
I20260807 22:45:07.382598 14264 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:07.382771 14264 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:07.382839 14264 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:07.382941 14264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:07.383823 14264 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:07.384176 14264 leader_election.cc:302] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa3616bab5040588a61480af614b866; no voters:
I20260807 22:45:07.384498 14264 leader_election.cc:288] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:45:07.384589 14273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:07.384879 14273 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 LEADER]: Becoming Leader. State: Replica: cfa3616bab5040588a61480af614b866, State: Running, Role: LEADER
I20260807 22:45:07.385260 14273 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:07.385735 14264 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:07.386509 14273 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfa3616bab5040588a61480af614b866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } } }
I20260807 22:45:07.386698 14273 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:07.386938 14273 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa3616bab5040588a61480af614b866. Latest consensus state: current_term: 1 leader_uuid: "cfa3616bab5040588a61480af614b866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } } }
I20260807 22:45:07.387087 14273 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:07.387390 14289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:07.387812 14264 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:45:07.396476 14289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:07.403527 14289 catalog_manager.cc:1347] Generated new cluster ID: 33aff20d578a4124ac3aaf69af8e0397
I20260807 22:45:07.403589 14289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:07.440412 14289 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:45:07.441854 14289 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:07.449538 14289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Generated new TSK 0
I20260807 22:45:07.450459 14289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:45:08.121173 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39777
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:08.253831 14335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:08.254125 14335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:08.254172 14335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:08.258172 14335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:08.258222 14335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:08.258280 14335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:08.263549 14335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.189.212:35993/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:08.264559 14335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:08.265623 14335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:08.276080 14345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:08.276613 14344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:08.276933 14335 server_base.cc:1034] running on GCE node
W20260807 22:45:08.277246 14347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:08.277721 14335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:08.278586 14335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:08.281936 14335 hybrid_clock.cc:648] HybridClock initialized: now 1786142708281546 us; error 397 us; skew 500 ppm
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:08.285570 14335 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:08.286762 14335 webserver.cc:460] Webserver started at http://127.26.189.193:40385/ using document root <none> and password file <none>
I20260807 22:45:08.287487 14335 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:08.287597 14335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:08.287865 14335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:08.303422 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "8199d1c0560e4964bd59824363ee1a4f"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "7c45e94574784357f76c7f62289834ed"
server_key_iv: "efd9fb1b49fb41da8f449fb8cac5e45f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:08.304033 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "8199d1c0560e4964bd59824363ee1a4f"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "7c45e94574784357f76c7f62289834ed"
server_key_iv: "efd9fb1b49fb41da8f449fb8cac5e45f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:08.307935 14335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:45:08.328225 14360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:08.330044 14335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:45:08.330201 14335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "8199d1c0560e4964bd59824363ee1a4f"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "7c45e94574784357f76c7f62289834ed"
server_key_iv: "efd9fb1b49fb41da8f449fb8cac5e45f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:08.330375 14335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:08.344802 14335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:08.348121 14335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:08.348359 14335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:08.349152 14335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:08.350209 14335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:08.350342 14335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:08.350466 14335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:08.350538 14335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:08.363523 14335 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:44893
I20260807 22:45:08.364674 14335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:08.366328 14553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:44893 every 8 connection(s)
I20260807 22:45:08.372951 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 14335
I20260807 22:45:08.373075 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:45:08.373342 27383 external_mini_cluster.cc:1428] Setting key 566fc36f5e52697ddd46554802b21ec7
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:08.401834 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:08.369581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:55379 (local address 127.26.189.254:39777)
0807 22:45:08.369853 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:08.369862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:08.380398 (+ 10536us) server_negotiation.cc:407] Connection header received
0807 22:45:08.381568 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:08.381579 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:08.381637 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:08.381742 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:08.391314 (+ 9572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.392169 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.393857 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.394099 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.394197 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:08.397508 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:08.397512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:08.397515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:08.397546 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:08.399432 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:08.400037 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:08.400041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:08.400043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:08.400147 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:08.400539 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:08.400542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:08.400544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:08.400765 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:08.400849 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:08.401512 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 22:45:08.401646 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":107,"threads_started":1}
I20260807 22:45:08.403723 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:08.368495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39777 (local address 127.26.189.193:55379)
0807 22:45:08.380231 (+ 11736us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:08.380294 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:08.381333 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:08.385625 (+ 4292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:08.385635 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:08.386005 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:08.386846 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:08.386868 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.392334 (+ 5466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.392337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:08.393745 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:08.393758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.394257 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.394261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:08.394505 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:08.395106 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:08.395125 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:08.397300 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:08.399572 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:08.399578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:08.399594 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:08.399920 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:08.400258 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:08.400261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:08.400263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:08.400433 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:08.400916 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:08.400922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:08.401235 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 22:45:08.402310 (+ 1075us) client_negotiation.cc:240] Negotiation successful
0807 22:45:08.402640 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11496,"thread_start_us":144,"threads_started":1}
I20260807 22:45:08.405369 14554 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39777
I20260807 22:45:08.405611 14554 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:08.406332 14554 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:08.408087 14204 ts_manager.cc:194] Registered new tserver with Master: 8199d1c0560e4964bd59824363ee1a4f (127.26.189.193:44893)
I20260807 22:45:08.409662 14204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:55379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:45:08.430945 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39777
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 22:45:08.603609 14569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:08.603960 14569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:08.604036 14569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:08.609795 14569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:08.609875 14569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:08.609958 14569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:08.617556 14569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.189.212:35993/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:08.619045 14569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:08.620441 14569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:08.632220 14577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:08.632210 14582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:08.632979 14578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:08.633033 14569 server_base.cc:1034] running on GCE node
I20260807 22:45:08.633579 14569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:08.634346 14569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:08.635519 14569 hybrid_clock.cc:648] HybridClock initialized: now 1786142708635493 us; error 42 us; skew 500 ppm
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:08.638705 14569 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:08.639540 14569 webserver.cc:460] Webserver started at http://127.26.189.194:35761/ using document root <none> and password file <none>
I20260807 22:45:08.640053 14569 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:08.640117 14569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:08.640301 14569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:08.656757 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "1b0f2647811e4b2a9887dd7fd8e9d823"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "d58d69a8a15c04abe33a5fdcf4cfa962"
server_key_iv: "42fd7b54d16327f685be5965ac221f30"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:08.657770 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "1b0f2647811e4b2a9887dd7fd8e9d823"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "d58d69a8a15c04abe33a5fdcf4cfa962"
server_key_iv: "42fd7b54d16327f685be5965ac221f30"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:08.668154 14569 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260807 22:45:08.690913 14596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:08.692716 14569 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 22:45:08.692862 14569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "1b0f2647811e4b2a9887dd7fd8e9d823"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "d58d69a8a15c04abe33a5fdcf4cfa962"
server_key_iv: "42fd7b54d16327f685be5965ac221f30"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:08.693020 14569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:08.719254 14569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:08.722960 14569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:08.723227 14569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:08.723981 14569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:08.725126 14569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:08.725252 14569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:08.725363 14569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:08.725435 14569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:08.738281 14569 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:44599
I20260807 22:45:08.738341 14789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:44599 every 8 connection(s)
I20260807 22:45:08.739557 14569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:08.745666 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 14569
I20260807 22:45:08.745796 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:45:08.746155 27383 external_mini_cluster.cc:1428] Setting key ffa743828b762e81c91075f6dee58348
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:08.772311 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:08.741614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53949 (local address 127.26.189.254:39777)
0807 22:45:08.741743 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:08.741748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:08.752265 (+ 10517us) server_negotiation.cc:407] Connection header received
0807 22:45:08.753612 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:08.753617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:08.753674 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:08.753726 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:08.755405 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.756249 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.757965 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.758353 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.758428 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:08.767956 (+ 9528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:08.767961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:08.767976 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:08.768009 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:08.769725 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:08.770436 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:08.770440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:08.770442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:08.770538 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:08.770952 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:08.770955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:08.770957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:08.771205 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:08.771287 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:08.771996 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 22:45:08.772138 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:45:08.774251 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:08.742126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39777 (local address 127.26.189.194:53949)
0807 22:45:08.752065 (+ 9939us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:08.752150 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:08.753343 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:08.754165 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:08.754175 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:08.754563 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:08.755231 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:08.755250 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.756407 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.756409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:08.757828 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:08.757840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:08.764788 (+ 6948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:08.764792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:08.765068 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:08.765829 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:08.765852 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:08.767700 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:08.769886 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:08.769893 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:08.769912 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:08.770286 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:08.770652 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:08.770655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:08.770657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:08.770841 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:08.771358 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:08.771365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:08.771697 (+ 332us) client_negotiation.cc:769] Sending connection context
0807 22:45:08.772779 (+ 1082us) client_negotiation.cc:240] Negotiation successful
0807 22:45:08.773146 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9665,"thread_start_us":172,"threads_started":1}
I20260807 22:45:08.776048 14795 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39777
I20260807 22:45:08.776304 14795 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:08.776889 14795 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:08.778079 14204 ts_manager.cc:194] Registered new tserver with Master: 1b0f2647811e4b2a9887dd7fd8e9d823 (127.26.189.194:44599)
I20260807 22:45:08.778865 14204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:53949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:45:08.795121 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39777
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:08.957937 14804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:08.958406 14804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:08.958534 14804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:08.964893 14804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:08.965034 14804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:08.965176 14804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:08.972965 14804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.189.212:35993/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:08.974547 14804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:08.975920 14804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:08.989280 14814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:08.989478 14810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:08.989373 14816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:08.989689 14804 server_base.cc:1034] running on GCE node
I20260807 22:45:08.990374 14804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:08.991150 14804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:08.992441 14804 hybrid_clock.cc:648] HybridClock initialized: now 1786142708992371 us; error 85 us; skew 500 ppm
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:08.996167 14804 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:08.997224 14804 webserver.cc:460] Webserver started at http://127.26.189.195:36631/ using document root <none> and password file <none>
I20260807 22:45:08.997884 14804 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:08.997957 14804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:08.998189 14804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:45:08 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:09.023752 14804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "08da2f6e6dcc46c68757e59448f9f164"
format_stamp: "Formatted at 2026-08-07 22:45:09 on dist-test-slave-4tdj"
server_key: "aa677c7a286b0c02f8f9812bc353722b"
server_key_iv: "3b5e6ed5917c32da3f62ac194d4f9756"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:09.032537 14804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "08da2f6e6dcc46c68757e59448f9f164"
format_stamp: "Formatted at 2026-08-07 22:45:09 on dist-test-slave-4tdj"
server_key: "aa677c7a286b0c02f8f9812bc353722b"
server_key_iv: "3b5e6ed5917c32da3f62ac194d4f9756"
server_key_version: "kuduclusterkey@0"
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:09.047788 14804 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.004s sys 0.000s
I20260807 22:45:09.069888 14832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.071694 14804 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260807 22:45:09.071939 14804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "08da2f6e6dcc46c68757e59448f9f164"
format_stamp: "Formatted at 2026-08-07 22:45:09 on dist-test-slave-4tdj"
server_key: "aa677c7a286b0c02f8f9812bc353722b"
server_key_iv: "3b5e6ed5917c32da3f62ac194d4f9756"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:09.072163 14804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:09.095201 14804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:09.099953 14804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:09.100196 14804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:09.100929 14804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:09.101984 14804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:09.102094 14804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.102185 14804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:09.102244 14804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.112600 14804 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:32897
I20260807 22:45:09.113768 14804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:09.118284 15020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:32897 every 8 connection(s)
I20260807 22:45:09.123984 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 14804
I20260807 22:45:09.124120 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:45:09.124419 27383 external_mini_cluster.cc:1428] Setting key 804d565002412628d2d3ab01e9795801
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142708, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:09.136613 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:09.121466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:46815 (local address 127.26.189.254:39777)
0807 22:45:09.121601 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:09.121606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:09.123533 (+ 1927us) server_negotiation.cc:407] Connection header received
0807 22:45:09.124875 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:09.124879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:09.124936 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:09.124988 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:09.126641 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.127417 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.129120 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.129453 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.129536 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:09.132426 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:09.132430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:09.132432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:09.132459 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:09.134112 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:09.134760 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:09.134764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:09.134766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:09.134851 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:09.135228 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:09.135232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:09.135234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:09.135489 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:09.135678 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:09.136308 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 22:45:09.136436 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:45:09.139436 15025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:09.121971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39777 (local address 127.26.189.195:46815)
0807 22:45:09.123353 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:09.123410 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:09.124614 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:09.125420 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:09.125430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:09.125774 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:09.126465 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:09.126487 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.127598 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.127601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:09.128984 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:09.128996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.129735 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.129738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:09.129923 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:09.130562 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:09.130582 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:09.132245 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:09.134252 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:09.134257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:09.134272 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:09.134639 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:09.134953 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:09.134956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:09.134957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:09.135123 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:09.135743 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:09.135750 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:09.136044 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 22:45:09.137391 (+ 1347us) client_negotiation.cc:240] Negotiation successful
0807 22:45:09.138382 (+ 991us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1168,"thread_start_us":132,"threads_started":1}
I20260807 22:45:09.140944 15022 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39777
I20260807 22:45:09.141197 15022 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:09.141734 15022 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:09.143129 14204 ts_manager.cc:194] Registered new tserver with Master: 08da2f6e6dcc46c68757e59448f9f164 (127.26.189.195:32897)
I20260807 22:45:09.143808 14204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:46815
I20260807 22:45:09.149904 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:09.151008 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 14335
I20260807 22:45:09.180955 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 14569
I20260807 22:45:09.199507 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 14804
I20260807 22:45:09.209514 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 14148
I20260807 22:45:09.226974 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39777
--webserver_interface=127.26.189.254
--webserver_port=33099
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39777
--ranger_config_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:09.337805 15026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:09.338052 15026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:09.338090 15026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:09.342043 15026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:09.342105 15026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:09.342123 15026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:09.342137 15026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:09.342150 15026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:09.347568 15026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39777
--ranger_config_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=33099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:09.348555 15026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:09.349521 15026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:09.357692 15035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:09.357841 15037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:09.358017 15026 server_base.cc:1034] running on GCE node
W20260807 22:45:09.357682 15039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:09.358711 15026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:09.360044 15026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:09.361233 15026 hybrid_clock.cc:648] HybridClock initialized: now 1786142709361212 us; error 32 us; skew 500 ppm
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:09.364531 15026 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:09.365605 15026 webserver.cc:460] Webserver started at http://127.26.189.254:33099/ using document root <none> and password file <none>
I20260807 22:45:09.366238 15026 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:09.366358 15026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:09.446079 15026 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:45:09.448329 15051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.449613 15026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:09.449750 15026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "cfa3616bab5040588a61480af614b866"
format_stamp: "Formatted at 2026-08-07 22:45:07 on dist-test-slave-4tdj"
server_key: "235a932d1c5546093b050ddfdfbb8cff"
server_key_iv: "044e96f5549f29981b3eaffb31377229"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:09.450284 15026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:09.482228 15026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:09.484704 15026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:09.484879 15026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:09.491869 15026 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:39777
I20260807 22:45:09.491888 15139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:39777 every 8 connection(s)
I20260807 22:45:09.492877 15026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:09.494854 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 15026
I20260807 22:45:09.495258 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:44893
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:39777
--webserver_port=40385
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:09.497045 15140 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:45:09.498646 15140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Bootstrap starting.
I20260807 22:45:09.514516 15140 log.cc:824] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:09.520787 15140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:09.521122 15140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Bootstrap complete.
I20260807 22:45:09.525053 15140 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:09.525450 15140 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa3616bab5040588a61480af614b866, State: Initialized, Role: FOLLOWER
I20260807 22:45:09.525888 15140 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:09.526002 15140 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:09.526068 15140 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:09.526157 15140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:45:09.527957 15140 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:09.528251 15140 leader_election.cc:302] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa3616bab5040588a61480af614b866; no voters:
I20260807 22:45:09.528522 15140 leader_election.cc:288] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:45:09.528651 15148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:45:09.528857 15148 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [term 2 LEADER]: Becoming Leader. State: Replica: cfa3616bab5040588a61480af614b866, State: Running, Role: LEADER
I20260807 22:45:09.529212 15148 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } }
I20260807 22:45:09.529724 15140 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:09.530589 15149 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cfa3616bab5040588a61480af614b866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } } }
I20260807 22:45:09.530716 15149 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:09.530965 15151 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa3616bab5040588a61480af614b866. Latest consensus state: current_term: 2 leader_uuid: "cfa3616bab5040588a61480af614b866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3616bab5040588a61480af614b866" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39777 } } }
I20260807 22:45:09.531038 15151 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:09.531064 15159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:09.534124 15159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:09.534629 15140 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:45:09.534816 15159 catalog_manager.cc:1259] Loaded cluster ID: 33aff20d578a4124ac3aaf69af8e0397
I20260807 22:45:09.534855 15159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:09.539628 15159 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:09.540800 15159 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Loaded TSK: 0
I20260807 22:45:09.541879 15159 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:45:09.637152 15145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:09.637526 15145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:09.637600 15145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:09.643500 15145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:09.643613 15145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:09.643723 15145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:09.651324 15145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:44893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=40385
--tserver_master_addrs=127.26.189.254:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:09.652647 15145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:09.653719 15145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:09.663105 15209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:09.663241 15204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:09.663568 15205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:09.663838 15145 server_base.cc:1034] running on GCE node
I20260807 22:45:09.664204 15145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:09.664829 15145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:09.666002 15145 hybrid_clock.cc:648] HybridClock initialized: now 1786142709665972 us; error 44 us; skew 500 ppm
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:09.668890 15145 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:09.669811 15145 webserver.cc:460] Webserver started at http://127.26.189.193:40385/ using document root <none> and password file <none>
I20260807 22:45:09.670369 15145 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:09.670457 15145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:09.691430 15145 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 22:45:09.694345 15221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.700415 15145 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260807 22:45:09.700558 15145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "8199d1c0560e4964bd59824363ee1a4f"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "7c45e94574784357f76c7f62289834ed"
server_key_iv: "efd9fb1b49fb41da8f449fb8cac5e45f"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:09.701305 15145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:09.722985 15145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:09.726090 15145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:09.726269 15145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:09.726935 15145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:09.727908 15145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:09.727962 15145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.728014 15145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:09.728068 15145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.739411 15145 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:44893
I20260807 22:45:09.739460 15393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:44893 every 8 connection(s)
I20260807 22:45:09.740485 15145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:09.743886 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 15145
I20260807 22:45:09.744419 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:44599
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:39777
--webserver_port=35761
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:09.760959 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:09.742765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39777 (local address 127.26.189.193:52991)
0807 22:45:09.743233 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:09.743284 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:09.744218 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:09.746917 (+ 2699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:09.746927 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:09.747291 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:09.747899 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:09.747917 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.749807 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.749812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:09.751208 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:09.751221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.752270 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.752274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:09.752465 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:09.753127 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:09.753148 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:09.754963 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:09.757434 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:09.757441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:09.757453 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:09.757800 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:09.758219 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:09.758222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:09.758224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:09.758428 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:09.759091 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:09.759098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:09.759390 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 22:45:09.759575 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 22:45:09.759913 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260807 22:45:09.763415 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:09.745276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:52991 (local address 127.26.189.254:39777)
0807 22:45:09.745782 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:09.745800 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:09.745836 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:45:09.746056 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:09.746066 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:09.746421 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:09.746804 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:09.748100 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.749643 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.751366 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:09.752088 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:09.754003 (+ 1915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:09.755204 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:09.755211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:09.755226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:09.755258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:09.757297 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:09.757948 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:09.757954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:09.757960 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:09.758101 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:09.758552 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:09.758554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:09.758555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:09.758956 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:09.762017 (+ 3061us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:09.762177 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 22:45:09.762509 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":141,"threads_started":1}
I20260807 22:45:09.766345 15397 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39777
I20260807 22:45:09.766585 15397 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:09.767191 15397 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
W20260807 22:45:09.769573 15397 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:39777 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:45:09.769707 15397 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:09.769874 15397 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:09.770892 15397 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:09.771059 15397 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
W20260807 22:45:09.771865 15397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:45:09.893496 15401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:09.893767 15401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:09.893812 15401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:09.898464 15401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:09.898537 15401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:09.898614 15401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:09.906066 15401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:44599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=35761
--tserver_master_addrs=127.26.189.254:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:09.907461 15401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:09.908880 15401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:09.920091 15410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:09.920105 15408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:09.920552 15414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:09.921154 15401 server_base.cc:1034] running on GCE node
I20260807 22:45:09.921563 15401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:09.922228 15401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:09.923413 15401 hybrid_clock.cc:648] HybridClock initialized: now 1786142709923388 us; error 40 us; skew 500 ppm
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:09.926101 15401 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:09.927038 15401 webserver.cc:460] Webserver started at http://127.26.189.194:35761/ using document root <none> and password file <none>
I20260807 22:45:09.927573 15401 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:09.927657 15401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:45:09 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:09.946682 15401 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:45:09.949389 15427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.950825 15401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:45:09.951009 15401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "1b0f2647811e4b2a9887dd7fd8e9d823"
format_stamp: "Formatted at 2026-08-07 22:45:08 on dist-test-slave-4tdj"
server_key: "d58d69a8a15c04abe33a5fdcf4cfa962"
server_key_iv: "42fd7b54d16327f685be5965ac221f30"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:09.951835 15401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:09.987510 15401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:09.991174 15401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:09.991460 15401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:09.992267 15401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:09.993378 15401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:09.993487 15401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:09.993597 15401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:09.993670 15401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:10.004103 15401 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:44599
I20260807 22:45:10.004272 15610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:44599 every 8 connection(s)
I20260807 22:45:10.005553 15401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:10.015566 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 15401
I20260807 22:45:10.016111 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.189.212:35993/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:32897
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:39777
--webserver_port=36631
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:10 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142709, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:10.024242 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:10.009287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:45165 (local address 127.26.189.254:39777)
0807 22:45:10.009456 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:10.009461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:10.010388 (+ 927us) server_negotiation.cc:407] Connection header received
0807 22:45:10.011353 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:10.011357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:10.011420 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:10.011530 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:10.012616 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.013411 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.015140 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.015456 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.015537 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:10.020210 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:10.020215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:10.020217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:10.020247 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:10.021756 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:10.022371 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:10.022375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:10.022377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:10.022461 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:10.022876 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:10.022880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:10.022882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:10.023117 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:10.023199 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:10.023913 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 22:45:10.024051 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:45:10.026175 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:10.009713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39777 (local address 127.26.189.194:45165)
0807 22:45:10.010206 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:10.010261 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:10.011125 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:10.011707 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:10.011714 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:10.011988 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:10.012453 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:10.012467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.013568 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.013572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:10.014978 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:10.014991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.015831 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.015835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:10.016034 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:10.018162 (+ 2128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:10.018184 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:10.019989 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:10.021886 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:10.021892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:10.021908 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:10.022230 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:10.022565 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:10.022567 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:10.022569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:10.022751 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:10.023270 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:10.023276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:10.023611 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 22:45:10.024714 (+ 1103us) client_negotiation.cc:240] Negotiation successful
0807 22:45:10.025074 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":123,"threads_started":1}
I20260807 22:45:10.027787 15612 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39777
I20260807 22:45:10.028035 15612 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.028648 15612 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
W20260807 22:45:10.030548 15612 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:39777 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:45:10.030687 15612 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.030864 15612 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:10.031879 15612 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.032047 15612 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
W20260807 22:45:10.033149 15612 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:45:10.147509 15618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:10.147982 15618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:10.148119 15618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:10.152583 15618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:10.152643 15618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:10.152705 15618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:10.158131 15618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.189.212:35993/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:32897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=36631
--tserver_master_addrs=127.26.189.254:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:10.159186 15618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:10.160233 15618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:10.170145 15626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:10.170146 15627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:10.170413 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:10.171233 15618 server_base.cc:1034] running on GCE node
I20260807 22:45:10.171696 15618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:10.172439 15618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:10.175711 15618 hybrid_clock.cc:648] HybridClock initialized: now 1786142710175678 us; error 42 us; skew 500 ppm
Aug 07 22:45:10 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142710, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:10.179050 15618 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:10.180171 15618 webserver.cc:460] Webserver started at http://127.26.189.195:36631/ using document root <none> and password file <none>
I20260807 22:45:10.180917 15618 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:10.181088 15618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:45:10 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142710, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:10.204083 15618 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:45:10.206879 15639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:10.208585 15618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:45:10.208714 15618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "08da2f6e6dcc46c68757e59448f9f164"
format_stamp: "Formatted at 2026-08-07 22:45:09 on dist-test-slave-4tdj"
server_key: "aa677c7a286b0c02f8f9812bc353722b"
server_key_iv: "3b5e6ed5917c32da3f62ac194d4f9756"
server_key_version: "kuduclusterkey@0"
I20260807 22:45:10.209437 15618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:10.232067 15618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:10.235378 15618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:10.235579 15618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:10.236303 15618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:10.237339 15618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:10.237411 15618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:10.237476 15618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:10.237509 15618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:10.249549 15618 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:32897
I20260807 22:45:10.249585 15815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:32897 every 8 connection(s)
I20260807 22:45:10.251209 15618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:10.257403 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 15618
Aug 07 22:45:10 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142710, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:10.267257 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:10.253055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:58349 (local address 127.26.189.254:39777)
0807 22:45:10.253172 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:10.253176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:10.254311 (+ 1135us) server_negotiation.cc:407] Connection header received
0807 22:45:10.255629 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:10.255633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:10.255689 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:10.255789 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:10.257255 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.258067 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.259840 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.260211 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.260294 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:10.263209 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:10.263213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:10.263216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:10.263243 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:10.264887 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:10.265518 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:10.265521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:10.265523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:10.265625 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:10.266012 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:10.266015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:10.266017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:10.266214 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:10.266321 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:10.266947 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 22:45:10.267076 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40,"spinlock_wait_cycles":896}
I20260807 22:45:10.268885 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:10.258451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.26.189.254:39777)
0807 22:45:10.259033 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:10.259036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:10.259045 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:45:10.259075 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:10.259077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:10.259138 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:10.259214 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:10.260131 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.261461 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.262785 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.263116 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.263215 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:10.265003 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:10.265007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:10.265009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:10.265033 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:10.266969 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:10.267577 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:10.267580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:10.267582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:10.267688 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:10.268013 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:10.268017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:10.268019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:10.268195 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:10.268380 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:10.268538 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 22:45:10.268658 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"spinlock_wait_cycles":21248,"thread_start_us":94,"threads_started":1}
I20260807 22:45:10.268877 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:10.253647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39777 (local address 127.26.189.195:58349)
0807 22:45:10.254139 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:10.254184 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:10.255377 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:10.255979 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:10.255986 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:10.256345 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:10.256991 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:10.257014 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.258332 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.258335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:10.259694 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:10.259706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:10.260497 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:10.260501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:10.260699 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:10.261325 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:10.261346 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:10.263007 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:10.265021 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:10.265027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:10.265043 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:10.265411 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:10.265723 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:10.265726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:10.265729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:10.265914 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:10.266344 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:10.266351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:10.266651 (+ 300us) client_negotiation.cc:769] Sending connection context
0807 22:45:10.267573 (+ 922us) client_negotiation.cc:240] Negotiation successful
0807 22:45:10.267888 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":151,"threads_started":1}
I20260807 22:45:10.270027 15816 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39777
I20260807 22:45:10.270455 15816 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.271018 15816 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
W20260807 22:45:10.272626 15816 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:39777 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:45:10.272730 15816 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.272886 15816 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:10.273653 15816 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.273802 15816 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
W20260807 22:45:10.274605 15816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 22:45:10 dist-test-slave-4tdj krb5kdc[13438](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142710, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:10.772105 15397 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:10.772423 15397 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:10.773617 15075 ts_manager.cc:194] Registered new tserver with Master: 8199d1c0560e4964bd59824363ee1a4f (127.26.189.193:44893)
I20260807 22:45:10.775309 15075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:52991
I20260807 22:45:11.033390 15612 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:11.033715 15612 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:11.034497 15075 ts_manager.cc:194] Registered new tserver with Master: 1b0f2647811e4b2a9887dd7fd8e9d823 (127.26.189.194:44599)
I20260807 22:45:11.035266 15075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:45165
Aug 07 22:45:11 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142710, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
Aug 07 22:45:11 dist-test-slave-4tdj krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142710, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for HTTP/127.26.189.212@KRBTEST.COM
I20260807 22:45:11.274835 15816 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:11.275130 15816 heartbeater.cc:511] Master 127.26.189.254:39777 requested a full tablet report, sending...
I20260807 22:45:11.275876 15075 ts_manager.cc:194] Registered new tserver with Master: 08da2f6e6dcc46c68757e59448f9f164 (127.26.189.195:32897)
I20260807 22:45:11.276615 15075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:58349
I20260807 22:45:11.281256 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:11.297260 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.283327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.26.189.254:39777)
0807 22:45:11.284523 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.284529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.284780 (+ 251us) server_negotiation.cc:407] Connection header received
0807 22:45:11.284915 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.284919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.284977 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.285246 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:11.286041 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.286792 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.287909 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.288219 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.288323 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.290328 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:11.290333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:11.290336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:11.290364 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:11.293237 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:11.294005 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:11.294008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:11.294010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:11.294253 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:11.294733 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:11.294736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:11.294738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:11.295100 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:11.295773 (+ 673us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.296945 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.297076 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":97,"threads_started":1}
I20260807 22:45:11.301359 15075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:45:11.306020 15075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:45:11.331934 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.319797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.26.189.193:44893)
0807 22:45:11.320641 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.320649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.326575 (+ 5926us) server_negotiation.cc:407] Connection header received
0807 22:45:11.326810 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.326814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.326960 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.327064 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:11.327513 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.328566 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.330189 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.331007 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.331102 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.331233 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.331663 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.331775 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":95,"threads_started":1}
I20260807 22:45:11.332690 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.319617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:44893 (local address 127.0.0.1:51144)
0807 22:45:11.326462 (+ 6845us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:11.326479 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:11.326667 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:11.327272 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:11.327275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:11.327294 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:11.327386 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.327392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.328731 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.328733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.330049 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.330060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.331333 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.331336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.331443 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.331481 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:45:11.332394 (+ 913us) client_negotiation.cc:240] Negotiation successful
0807 22:45:11.332534 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6778,"thread_start_us":86,"threads_started":1}
I20260807 22:45:11.336735 15299 tablet_service.cc:1467] Processing CreateTablet for tablet 51b089025b2e4e789abd8db359a52530 (DEFAULT_TABLE table=test-table [id=848c18ab22914294851c68c3aa389ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:11.338058 15299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b089025b2e4e789abd8db359a52530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:11.347210 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.318709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:44599 (local address 127.0.0.1:57084)
0807 22:45:11.320044 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:11.320056 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:11.320181 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:11.327074 (+ 6893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:11.327081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:11.327108 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:11.327254 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.327261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.332966 (+ 5705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.332969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.336956 (+ 3987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.336969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.346779 (+ 9810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.346783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.346878 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.346903 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:45:11.346978 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:45:11.347061 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"thread_start_us":99,"threads_started":1}
I20260807 22:45:11.347952 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.318864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.26.189.194:44599)
0807 22:45:11.326557 (+ 7693us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.326564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.326584 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:45:11.326666 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.326672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.326903 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.327065 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:11.327403 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.332829 (+ 5426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.337115 (+ 4286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.342237 (+ 5122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.346555 (+ 4318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.346718 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.347705 (+ 987us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.347817 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7576,"thread_start_us":75,"threads_started":1}
I20260807 22:45:11.349970 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.317973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:32897 (local address 127.0.0.1:60722)
0807 22:45:11.319279 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:11.319334 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:11.320352 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:11.327840 (+ 7488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:11.327844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:11.327863 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:11.327954 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.327959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.344110 (+ 16151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.344114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.345323 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.345335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.349583 (+ 4248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.349587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.349690 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.349716 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:45:11.349774 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:45:11.349845 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":104,"threads_started":1}
I20260807 22:45:11.350773 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.318077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.26.189.195:32897)
0807 22:45:11.327423 (+ 9346us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.327430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.327451 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:45:11.327523 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.327528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.327715 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.327862 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:11.332137 (+ 4275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.341427 (+ 9290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.345493 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.346370 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.350352 (+ 3982us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.350489 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.350540 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.350630 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9236,"thread_start_us":82,"threads_started":1}
I20260807 22:45:11.351001 15507 tablet_service.cc:1467] Processing CreateTablet for tablet 51b089025b2e4e789abd8db359a52530 (DEFAULT_TABLE table=test-table [id=848c18ab22914294851c68c3aa389ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:11.352082 15507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b089025b2e4e789abd8db359a52530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:11.353495 15884 tablet_bootstrap.cc:492] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: Bootstrap starting.
I20260807 22:45:11.355407 15685 tablet_service.cc:1467] Processing CreateTablet for tablet 51b089025b2e4e789abd8db359a52530 (DEFAULT_TABLE table=test-table [id=848c18ab22914294851c68c3aa389ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:11.356401 15685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b089025b2e4e789abd8db359a52530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:11.359879 15884 tablet_bootstrap.cc:654] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:11.360785 15884 log.cc:824] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:11.367192 15887 tablet_bootstrap.cc:492] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Bootstrap starting.
I20260807 22:45:11.369041 15888 tablet_bootstrap.cc:492] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Bootstrap starting.
I20260807 22:45:11.371243 15888 tablet_bootstrap.cc:654] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:11.372215 15888 log.cc:824] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:11.374776 15888 tablet_bootstrap.cc:492] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: No bootstrap required, opened a new log
I20260807 22:45:11.374975 15888 ts_tablet_manager.cc:1397] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:45:11.375334 15887 tablet_bootstrap.cc:654] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:11.376083 15887 log.cc:824] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:11.376402 15884 tablet_bootstrap.cc:492] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: No bootstrap required, opened a new log
I20260807 22:45:11.376582 15884 ts_tablet_manager.cc:1397] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.006s
I20260807 22:45:11.379851 15888 raft_consensus.cc:348] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.380237 15888 raft_consensus.cc:374] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:11.380333 15888 raft_consensus.cc:729] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0f2647811e4b2a9887dd7fd8e9d823, State: Initialized, Role: FOLLOWER
I20260807 22:45:11.380369 15884 raft_consensus.cc:348] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.380668 15884 raft_consensus.cc:374] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:11.380759 15884 raft_consensus.cc:729] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8199d1c0560e4964bd59824363ee1a4f, State: Initialized, Role: FOLLOWER
I20260807 22:45:11.380903 15888 consensus_queue.cc:260] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.381328 15884 consensus_queue.cc:260] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.382092 15888 ts_tablet_manager.cc:1428] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 22:45:11.382426 15884 ts_tablet_manager.cc:1428] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:45:11.382547 15612 heartbeater.cc:503] Master 127.26.189.254:39777 was elected leader, sending a full tablet report...
I20260807 22:45:11.382822 15397 heartbeater.cc:503] Master 127.26.189.254:39777 was elected leader, sending a full tablet report...
I20260807 22:45:11.394186 15887 tablet_bootstrap.cc:492] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: No bootstrap required, opened a new log
I20260807 22:45:11.394536 15887 ts_tablet_manager.cc:1397] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.006s
I20260807 22:45:11.398615 15887 raft_consensus.cc:348] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.398962 15887 raft_consensus.cc:374] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:11.399053 15887 raft_consensus.cc:729] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08da2f6e6dcc46c68757e59448f9f164, State: Initialized, Role: FOLLOWER
I20260807 22:45:11.399657 15887 consensus_queue.cc:260] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.400892 15887 ts_tablet_manager.cc:1428] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:45:11.401278 15816 heartbeater.cc:503] Master 127.26.189.254:39777 was elected leader, sending a full tablet report...
W20260807 22:45:11.493552 15398 tablet.cc:2369] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:45:11.502969 15819 tablet.cc:2369] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:45:11.510162 15613 tablet.cc:2369] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:11.539340 15891 raft_consensus.cc:482] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:11.539597 15891 raft_consensus.cc:504] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.540951 15891 leader_election.cc:288] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08da2f6e6dcc46c68757e59448f9f164 (127.26.189.195:32897), 8199d1c0560e4964bd59824363ee1a4f (127.26.189.193:44893)
I20260807 22:45:11.547439 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.542628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:49467 (local address 127.26.189.195:32897)
0807 22:45:11.542731 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.542735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.542966 (+ 231us) server_negotiation.cc:407] Connection header received
0807 22:45:11.543155 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.543159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.543206 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.543250 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:11.543665 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.544451 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.546090 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.546730 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.546829 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.546882 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.547226 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.547307 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:45:11.548040 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.541397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:32897 (local address 127.26.189.194:49467)
0807 22:45:11.542885 (+ 1488us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:11.542898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:11.543043 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:11.543380 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:11.543383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:11.543399 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:11.543533 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.543538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.544590 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.544593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.545934 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.545947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.546976 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.546979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.547087 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.547110 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:45:11.547815 (+ 705us) client_negotiation.cc:240] Negotiation successful
0807 22:45:11.547895 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1422,"thread_start_us":121,"threads_started":1}
I20260807 22:45:11.549160 15717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b089025b2e4e789abd8db359a52530" candidate_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08da2f6e6dcc46c68757e59448f9f164" is_pre_election: true
I20260807 22:45:11.549494 15717 raft_consensus.cc:2454] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b0f2647811e4b2a9887dd7fd8e9d823 in term 0.
I20260807 22:45:11.550009 15432 leader_election.cc:302] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08da2f6e6dcc46c68757e59448f9f164, 1b0f2647811e4b2a9887dd7fd8e9d823; no voters:
I20260807 22:45:11.550359 15891 raft_consensus.cc:2781] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:11.550442 15891 raft_consensus.cc:482] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:11.550483 15891 raft_consensus.cc:3037] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:11.551792 15891 raft_consensus.cc:504] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.552098 15891 leader_election.cc:288] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [CANDIDATE]: Term 1 election: Requested vote from peers 08da2f6e6dcc46c68757e59448f9f164 (127.26.189.195:32897), 8199d1c0560e4964bd59824363ee1a4f (127.26.189.193:44893)
I20260807 22:45:11.552490 15717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b089025b2e4e789abd8db359a52530" candidate_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08da2f6e6dcc46c68757e59448f9f164"
I20260807 22:45:11.552631 15717 raft_consensus.cc:3037] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:11.553911 15717 raft_consensus.cc:2454] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b0f2647811e4b2a9887dd7fd8e9d823 in term 1.
I20260807 22:45:11.554284 15432 leader_election.cc:302] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08da2f6e6dcc46c68757e59448f9f164, 1b0f2647811e4b2a9887dd7fd8e9d823; no voters:
I20260807 22:45:11.554482 15891 raft_consensus.cc:2781] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:11.554697 15891 raft_consensus.cc:686] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 1 LEADER]: Becoming Leader. State: Replica: 1b0f2647811e4b2a9887dd7fd8e9d823, State: Running, Role: LEADER
I20260807 22:45:11.555013 15891 consensus_queue.cc:237] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } }
I20260807 22:45:11.557931 15075 catalog_manager.cc:5495] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b0f2647811e4b2a9887dd7fd8e9d823 (127.26.189.194). New cstate: current_term: 1 leader_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 44599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 } health_report { overall_health: UNKNOWN } } }
I20260807 22:45:11.560375 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.542328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:46145 (local address 127.26.189.193:44893)
0807 22:45:11.542456 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.542460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.548499 (+ 6039us) server_negotiation.cc:407] Connection header received
0807 22:45:11.549095 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.549099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.549156 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.549209 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:11.552365 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.553082 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.559167 (+ 6085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.559652 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.559737 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.559812 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.560171 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.560258 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:45:11.561185 15894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.541786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:44893 (local address 127.26.189.194:46145)
0807 22:45:11.548406 (+ 6620us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:11.548420 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:11.548899 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:11.549431 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:11.549434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:11.549453 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:11.549559 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.549564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.553261 (+ 3697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.553264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.559023 (+ 5759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:11.559035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.559933 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.559936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:11.560026 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.560049 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:45:11.560831 (+ 782us) client_negotiation.cc:240] Negotiation successful
0807 22:45:11.560960 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6553,"thread_start_us":86,"threads_started":1}
I20260807 22:45:11.561765 15327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b089025b2e4e789abd8db359a52530" candidate_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8199d1c0560e4964bd59824363ee1a4f" is_pre_election: true
I20260807 22:45:11.561769 15323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b089025b2e4e789abd8db359a52530" candidate_uuid: "1b0f2647811e4b2a9887dd7fd8e9d823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8199d1c0560e4964bd59824363ee1a4f"
I20260807 22:45:11.562007 15323 raft_consensus.cc:3037] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:11.563423 15323 raft_consensus.cc:2454] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b0f2647811e4b2a9887dd7fd8e9d823 in term 1.
I20260807 22:45:11.573933 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.569078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.26.189.194:44599)
0807 22:45:11.569213 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:11.569218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:11.569458 (+ 240us) server_negotiation.cc:407] Connection header received
0807 22:45:11.569528 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:11.569531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:11.569587 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:11.569677 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:11.570015 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.570800 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.571994 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:11.572326 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:11.572414 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:11.572797 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:11.573309 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:11.573417 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:11.573724 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 22:45:11.573780 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:45:11.580870 15717 raft_consensus.cc:1261] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 1 FOLLOWER]: Refusing update from remote peer 1b0f2647811e4b2a9887dd7fd8e9d823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:11.581547 15897 consensus_queue.cc:1035] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08da2f6e6dcc46c68757e59448f9f164" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 32897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:11.581687 15323 raft_consensus.cc:1261] T 51b089025b2e4e789abd8db359a52530 P 8199d1c0560e4964bd59824363ee1a4f [term 1 FOLLOWER]: Refusing update from remote peer 1b0f2647811e4b2a9887dd7fd8e9d823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:11.586618 15897 consensus_queue.cc:1035] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8199d1c0560e4964bd59824363ee1a4f" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 44893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:11.595772 15917 mvcc.cc:205] Tried to move back new op lower bound from 7316040546626244608 to 7316040546531790848. Current Snapshot: MvccSnapshot[applied={T|T < 7316040546626244608}]
I20260807 22:45:11.606618 15074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:11.606793 15074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:11.609287 15074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfa3616bab5040588a61480af614b866: Sending DeleteTablet for 3 replicas of tablet 51b089025b2e4e789abd8db359a52530
I20260807 22:45:11.610090 15685 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b089025b2e4e789abd8db359a52530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:11 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:60722
I20260807 22:45:11.610469 15507 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b089025b2e4e789abd8db359a52530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:11 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:57084
I20260807 22:45:11.610471 15299 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b089025b2e4e789abd8db359a52530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:11 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:51144
I20260807 22:45:11.610769 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 15145
I20260807 22:45:11.622057 15934 tablet_replica.cc:330] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: stopping tablet replica
I20260807 22:45:11.622431 15934 raft_consensus.cc:2229] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:45:11.622924 15934 raft_consensus.cc:2258] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:11.624574 15934 ts_tablet_manager.cc:1905] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:11.627435 15934 ts_tablet_manager.cc:1918] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:11.627591 15934 log.cc:1196] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/51b089025b2e4e789abd8db359a52530
I20260807 22:45:11.627983 15934 ts_tablet_manager.cc:1939] T 51b089025b2e4e789abd8db359a52530 P 1b0f2647811e4b2a9887dd7fd8e9d823: Deleting consensus metadata
I20260807 22:45:11.629242 15053 catalog_manager.cc:4841] TS 1b0f2647811e4b2a9887dd7fd8e9d823 (127.26.189.194:44599): tablet 51b089025b2e4e789abd8db359a52530 (table test-table [id=848c18ab22914294851c68c3aa389ccc]) successfully deleted
I20260807 22:45:11.629506 15931 tablet_replica.cc:330] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: stopping tablet replica
I20260807 22:45:11.629864 15931 raft_consensus.cc:2229] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:11.630170 15931 raft_consensus.cc:2258] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 22:45:11.631966 15056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:44893 (error 108)
I20260807 22:45:11.632133 15931 ts_tablet_manager.cc:1905] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:11.633329 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 15401
I20260807 22:45:11.633352 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:11.633081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:44893 (local address 127.0.0.1:51154)
0807 22:45:11.633196 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:11.633266 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:44893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:45:11.654886 15056 catalog_manager.cc:4584] TS 8199d1c0560e4964bd59824363ee1a4f (127.26.189.193:44893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51b089025b2e4e789abd8db359a52530: Network error: Client connection negotiation failed: client connection to 127.26.189.193:44893: connect: Connection refused (error 111)
I20260807 22:45:11.659264 15931 ts_tablet_manager.cc:1918] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:11.659441 15931 log.cc:1196] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/51b089025b2e4e789abd8db359a52530
I20260807 22:45:11.659818 15931 ts_tablet_manager.cc:1939] T 51b089025b2e4e789abd8db359a52530 P 08da2f6e6dcc46c68757e59448f9f164: Deleting consensus metadata
I20260807 22:45:11.661274 15056 catalog_manager.cc:4841] TS 08da2f6e6dcc46c68757e59448f9f164 (127.26.189.195:32897): tablet 51b089025b2e4e789abd8db359a52530 (table test-table [id=848c18ab22914294851c68c3aa389ccc]) successfully deleted
I20260807 22:45:11.663219 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 15618
I20260807 22:45:11.672246 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 15026
I20260807 22:45:11.689544 27383 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:45:12.033510 27383 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:45:12.033623 27383 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:45:12.387398 27383 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:45:12.387 UTC [13484] LOG: received smart shutdown request
2026-08-07 22:45:12.391 UTC [13484] LOG: background worker "logical replication launcher" (PID 13498) exited with exit code 1
2026-08-07 22:45:12.392 UTC [13491] LOG: shutting down
2026-08-07 22:45:12.525 UTC [13484] LOG: database system is shut down
2026-08-07T22:45:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48183 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:45:12 dist-test-slave-4tdj krb5kdc[15950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:45:12 dist-test-slave-4tdj krb5kdc[15950](info): set up 2 sockets
Aug 07 22:45:12 dist-test-slave-4tdj krb5kdc[15950](info): commencing operation
krb5kdc: starting...
W20260807 22:45:13.758809 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:45:13 dist-test-slave-4tdj krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:45:13.880282 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:45861
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:45861,127.26.189.253:37099,127.26.189.252:41133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:13.990247 15968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:13.990523 15968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:13.990579 15968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:13.994511 15968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:13.994568 15968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:13.994593 15968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:13.994616 15968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:13.994645 15968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:13.999743 15968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:45861,127.26.189.253:37099,127.26.189.252:41133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:14.000754 15968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:14.001771 15968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:14.010061 15976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:14.010126 15975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:14.010102 15979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:14.010427 15968 server_base.cc:1034] running on GCE node
I20260807 22:45:14.010977 15968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:14.012032 15968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:14.013202 15968 hybrid_clock.cc:648] HybridClock initialized: now 1786142714013186 us; error 28 us; skew 500 ppm
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:14.016029 15968 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:14.016877 15968 webserver.cc:460] Webserver started at http://127.26.189.254:42255/ using document root <none> and password file <none>
I20260807 22:45:14.017400 15968 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:14.017460 15968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:14.017660 15968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:14.019714 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "301b7179dab74b4ca7c410aa4596f91b"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "765f14604035fcbd614132a83353e8c6"
server_key_iv: "731694a0d089d77f80f96c0b823207b0"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.020257 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "301b7179dab74b4ca7c410aa4596f91b"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "765f14604035fcbd614132a83353e8c6"
server_key_iv: "731694a0d089d77f80f96c0b823207b0"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.023850 15968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:45:14.026149 15988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:14.027355 15968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:14.027454 15968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "301b7179dab74b4ca7c410aa4596f91b"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "765f14604035fcbd614132a83353e8c6"
server_key_iv: "731694a0d089d77f80f96c0b823207b0"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.027535 15968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:14.043555 15968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:14.046406 15968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:14.046557 15968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:14.053332 15968 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:45861
I20260807 22:45:14.053594 16074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:45861 every 8 connection(s)
I20260807 22:45:14.054258 15968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:14.056548 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 15968
I20260807 22:45:14.056694 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:45:14.057009 27383 external_mini_cluster.cc:1428] Setting key 5c753e4a6a1fd6974b6b18821979c2ec
I20260807 22:45:14.057165 16075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:14.061318 16075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.189.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.253@KRBTEST.COM" created.
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:14.075901 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.063717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39280 (local address 127.26.189.254:45861)
0807 22:45:14.064129 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.064147 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.064407 (+ 260us) server_negotiation.cc:407] Connection header received
0807 22:45:14.065418 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.065422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.065616 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.065791 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.066691 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.067548 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.068574 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.068930 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.069074 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.071501 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.071507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.071516 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.071536 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.072994 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.073415 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.073420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.073421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.073516 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.073788 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.073791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.073793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.074143 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.074258 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.074648 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.074890 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
I20260807 22:45:14.075901 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.063718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45861 (local address 127.0.0.1:39280)
0807 22:45:14.064271 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.064308 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.065119 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.065804 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.065810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.066229 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.066504 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.066519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.067677 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.067681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.068460 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.068470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.069080 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.069084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.069197 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.069774 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.069795 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.071371 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.073096 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.073104 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.073113 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.073318 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.073597 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.073600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.073602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.073709 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.074245 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.074251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.074419 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.074566 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.074948 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":91,"spinlock_wait_cycles":768,"thread_start_us":155,"threads_started":1}
I20260807 22:45:14.078210 16075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:45:14.078908 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.078637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52930)
0807 22:45:14.078771 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.078845 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 22:45:14.079097 15989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
I20260807 22:45:14.079883 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.079695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52932)
0807 22:45:14.079785 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.079828 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:45:14.080055 16075 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:37099: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab.
Entry for principal kudu/127.26.189.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab.
I20260807 22:45:14.083325 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:37099
--webserver_interface=127.26.189.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab
--principal=kudu/127.26.189.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:45861,127.26.189.253:37099,127.26.189.252:41133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:14.129288 16075 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } attempt: 1
I20260807 22:45:14.130219 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.129909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52944)
0807 22:45:14.130060 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.130126 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 22:45:14.130940 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.130729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52954)
0807 22:45:14.130840 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.130885 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 22:45:14.131055 16075 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:37099: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
W20260807 22:45:14.194022 16086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:14.194272 16086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:14.194331 16086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:14.198601 16086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:14.198657 16086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:14.198674 16086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:14.198689 16086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:14.198704 16086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:14.199193 16075 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } attempt: 2
I20260807 22:45:14.200024 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.199732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52956)
0807 22:45:14.199881 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.199944 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 22:45:14.200695 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.200492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52962)
0807 22:45:14.200586 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.200638 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 22:45:14.200835 16075 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:37099: Network error: Client connection negotiation failed: client connection to 127.26.189.253:37099: connect: Connection refused (error 111)
I20260807 22:45:14.204258 16086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:45861,127.26.189.253:37099,127.26.189.252:41133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab
--principal=kudu/127.26.189.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:37099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.189.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:14.205250 16086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:14.206380 16086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:14.215068 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:14.215099 16096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:14.215236 16086 server_base.cc:1034] running on GCE node
W20260807 22:45:14.215070 16094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:14.215936 16086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:14.217005 16086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:14.218159 16086 hybrid_clock.cc:648] HybridClock initialized: now 1786142714218134 us; error 38 us; skew 500 ppm
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:14.220739 16086 init.cc:377] Logged in from keytab as kudu/127.26.189.253@KRBTEST.COM (short username kudu)
I20260807 22:45:14.221561 16086 webserver.cc:460] Webserver started at http://127.26.189.253:36003/ using document root <none> and password file <none>
I20260807 22:45:14.222048 16086 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:14.222103 16086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:14.222280 16086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:14.224323 16086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data/instance:
uuid: "3089c8968b6b4427a10be299fef74a7f"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "58cd1d66f2c3d0432cb5d099427a3058"
server_key_iv: "413d500e36d91ad9cd9d28d7b558f764"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.224850 16086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal/instance:
uuid: "3089c8968b6b4427a10be299fef74a7f"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "58cd1d66f2c3d0432cb5d099427a3058"
server_key_iv: "413d500e36d91ad9cd9d28d7b558f764"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.228466 16086 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:45:14.230762 16106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:14.231951 16086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:14.232069 16086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal
uuid: "3089c8968b6b4427a10be299fef74a7f"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "58cd1d66f2c3d0432cb5d099427a3058"
server_key_iv: "413d500e36d91ad9cd9d28d7b558f764"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.232172 16086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:14.250456 16086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:14.253227 16086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:14.253392 16086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:14.260354 16086 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.253:37099
I20260807 22:45:14.260633 16175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.253:37099 every 8 connection(s)
I20260807 22:45:14.261268 16086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
I20260807 22:45:14.263949 16177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:14.267643 16177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:45:14.269766 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 16086
I20260807 22:45:14.269901 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-1/wal/instance
I20260807 22:45:14.270224 27383 external_mini_cluster.cc:1428] Setting key 72e7374cd8e9fa69069ffab368501a72
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.189.252@KRBTEST.COM; defaulting to no policy
I20260807 22:45:14.280367 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.269402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.26.189.254:45861)
0807 22:45:14.269536 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.269540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.270513 (+ 973us) server_negotiation.cc:407] Connection header received
0807 22:45:14.271385 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.271389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.271444 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.271517 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.272621 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.273164 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.274253 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.274544 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.274636 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.276941 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.276950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.276952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.276978 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.278415 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.278933 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.278937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.278939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.279026 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.279374 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.279377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.279378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.279566 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.279677 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.280136 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.280239 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.252@KRBTEST.COM" created.
I20260807 22:45:14.281276 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.269847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45861 (local address 127.0.0.1:39284)
0807 22:45:14.270377 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.270421 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.271193 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.271704 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.271711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.272027 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.272482 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.272495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.273291 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.273294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.274140 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.274149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.274693 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.274697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.274853 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.275424 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.275440 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.276781 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.278548 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.278554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.278568 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.278837 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.279112 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.279115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.279117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.279277 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.279687 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.279692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.279930 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.280095 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.280379 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":140,"threads_started":1}
I20260807 22:45:14.282750 16177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:45:14.291908 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.283230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52976)
0807 22:45:14.283360 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.283375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.283456 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.283984 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.283987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.284130 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.284220 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.284225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.285571 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.285574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.286118 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.286127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.286937 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.286940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.287030 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.287503 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.287516 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.288675 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.290481 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.290484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.290486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.290661 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.290917 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.290920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.290921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.291016 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.291448 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.291451 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.291572 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.291669 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.291783 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 22:45:14.292001 16075 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } attempt: 3
I20260807 22:45:14.292183 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.283354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52976 (local address 127.26.189.253:37099)
0807 22:45:14.283644 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.283653 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.283672 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:45:14.283707 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.283711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.283851 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.283988 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.284327 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.285439 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.286233 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.286829 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.286945 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.288816 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.288828 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.288843 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.288871 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.290366 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.290762 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.290768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.290769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.290847 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.291091 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.291093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.291095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.291336 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.291483 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.291763 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.291882 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":197,"thread_start_us":110,"threads_started":1}
I20260807 22:45:14.294255 16177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:45:14.294998 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.294697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45232)
0807 22:45:14.294878 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.294944 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260807 22:45:14.295195 16109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
I20260807 22:45:14.296115 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.295915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45234)
0807 22:45:14.296011 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.296056 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 22:45:14.296309 16177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:41133: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab.
Entry for principal kudu/127.26.189.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab.
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:45:14.297508 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:41133
--webserver_interface=127.26.189.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab
--principal=kudu/127.26.189.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:45861,127.26.189.253:37099,127.26.189.252:41133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:14.300424 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.292625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52992 (local address 127.26.189.253:37099)
0807 22:45:14.292754 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.292759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.292773 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:45:14.292827 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.292829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.292864 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.292924 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.293400 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.293902 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.295280 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.295599 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.295696 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.297742 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.297746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.297748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.297771 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.298976 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.299444 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.299447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.299448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.299522 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.299773 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.299775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.299776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.299922 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.299988 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.300246 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.300332 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:45:14.300603 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.292457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:52992)
0807 22:45:14.292586 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.292608 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.292687 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.292966 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.292969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.293134 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.293255 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.293264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.294017 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.294020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.295118 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.295132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.295708 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.295712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.295826 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.296308 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.296324 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.297613 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.299084 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.299087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.299089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.299343 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.299599 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.299602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.299604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.299696 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.299993 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.299996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.300109 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.300216 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.300362 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 22:45:14.301658 16075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:45:14.302374 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.302114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45240)
0807 22:45:14.302225 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.302282 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 22:45:14.303063 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.302837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45252)
0807 22:45:14.302964 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.303017 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 22:45:14.303169 16075 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:41133: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
I20260807 22:45:14.334314 16075 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } attempt: 1
I20260807 22:45:14.335057 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.334785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45268)
0807 22:45:14.334923 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.334981 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 22:45:14.335791 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.335509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45270)
0807 22:45:14.335650 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.335715 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 22:45:14.335947 16075 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:41133: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
I20260807 22:45:14.345563 16177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } attempt: 1
I20260807 22:45:14.346328 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.346031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45286)
0807 22:45:14.346164 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.346220 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 22:45:14.346971 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.346792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45296)
0807 22:45:14.346877 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.346919 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:45:14.347081 16177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:41133: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
W20260807 22:45:14.407904 16185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:14.408143 16185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:14.408180 16185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 22:45:14.411103 16075 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } attempt: 2
I20260807 22:45:14.411907 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.411616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45300)
0807 22:45:14.411771 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.411829 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 22:45:14.412335 16185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:14.412397 16185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:14.412424 16185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:14.412458 16185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:14.412485 16185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:14.412689 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.412412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45302)
0807 22:45:14.412584 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.412633 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:45:14.412808 16075 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:41133: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
I20260807 22:45:14.415215 16177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } attempt: 2
I20260807 22:45:14.415831 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.415610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45310)
0807 22:45:14.415721 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.415768 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:45:14.416412 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.416227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45316)
0807 22:45:14.416320 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.416362 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 22:45:14.416520 16177 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:41133: Network error: Client connection negotiation failed: client connection to 127.26.189.252:41133: connect: Connection refused (error 111)
I20260807 22:45:14.417999 16185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:45861,127.26.189.253:37099,127.26.189.252:41133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab
--principal=kudu/127.26.189.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:41133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.189.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:14.419008 16185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:14.420081 16185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:14.428164 16196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:14.428294 16185 server_base.cc:1034] running on GCE node
W20260807 22:45:14.428164 16191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:14.428160 16192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:14.428941 16185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:14.429996 16185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:14.431144 16185 hybrid_clock.cc:648] HybridClock initialized: now 1786142714431127 us; error 28 us; skew 500 ppm
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:14.433763 16185 init.cc:377] Logged in from keytab as kudu/127.26.189.252@KRBTEST.COM (short username kudu)
I20260807 22:45:14.434623 16185 webserver.cc:460] Webserver started at http://127.26.189.252:36499/ using document root <none> and password file <none>
I20260807 22:45:14.435138 16185 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:14.435204 16185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:14.435410 16185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:14.437482 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data/instance:
uuid: "39c0c772ee9940efa9765981c34b3b3b"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "abd96afff57645cedf9d77399d9c91c8"
server_key_iv: "10eea0c1d8091eb8a4f2fa736d1a9223"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.438023 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal/instance:
uuid: "39c0c772ee9940efa9765981c34b3b3b"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "abd96afff57645cedf9d77399d9c91c8"
server_key_iv: "10eea0c1d8091eb8a4f2fa736d1a9223"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.441555 16185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:45:14.443867 16206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:14.445082 16185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:14.445217 16185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal
uuid: "39c0c772ee9940efa9765981c34b3b3b"
format_stamp: "Formatted at 2026-08-07 22:45:14 on dist-test-slave-4tdj"
server_key: "abd96afff57645cedf9d77399d9c91c8"
server_key_iv: "10eea0c1d8091eb8a4f2fa736d1a9223"
server_key_version: "encryptionkey@0"
I20260807 22:45:14.445307 16185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:14.459309 16185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:14.461921 16185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:14.462091 16185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:14.468631 16185 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.252:41133
I20260807 22:45:14.468883 16286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.252:41133 every 8 connection(s)
I20260807 22:45:14.469514 16185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
I20260807 22:45:14.472167 16287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:14.474663 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 16185
I20260807 22:45:14.474805 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142556986508-27383-0/minicluster-data/master-2/wal/instance
I20260807 22:45:14.475069 27383 external_mini_cluster.cc:1428] Setting key 81f340d5df5c6fe4f5b75d13b7b6bbe2
I20260807 22:45:14.475982 16287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:45:14.487957 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.477676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.26.189.254:45861)
0807 22:45:14.477801 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.477806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.478067 (+ 261us) server_negotiation.cc:407] Connection header received
0807 22:45:14.478339 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.478343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.478398 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.478482 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.479311 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.480071 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.481404 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.481746 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.481846 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.484809 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.484813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.484815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.484838 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.486246 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.486679 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.486683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.486684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.486759 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.487033 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.487036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.487038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.487249 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.487335 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.487676 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.487793 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:45:14.488732 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.477588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:45861 (local address 127.0.0.1:39300)
0807 22:45:14.477942 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.477988 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.478158 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.478558 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.478564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.478993 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.479174 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.479184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.480163 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.480166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.481296 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.481307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.481892 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.481895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.482012 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.482684 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.482703 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.484663 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.486353 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.486361 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.486371 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.486580 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.486838 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.486841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.486843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.486948 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.487349 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.487356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.487479 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.487627 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.487924 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260807 22:45:14.488732 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.476255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45330 (local address 127.26.189.252:41133)
0807 22:45:14.476654 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.476669 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.476703 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:45:14.477342 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.477363 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.477770 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.478077 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.478946 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.480229 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.481327 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.481863 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.482002 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.484785 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.484790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.484799 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.484820 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.486341 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.486737 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.486742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.486744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.486840 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.487084 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.487087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.487089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.487422 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.487602 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.487812 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.488217 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":265,"spinlock_wait_cycles":9728,"thread_start_us":118,"threads_started":1}
I20260807 22:45:14.490422 16287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:14.497895 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.490880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:37099 (local address 127.0.0.1:53000)
0807 22:45:14.491097 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.491111 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.491179 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.491388 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.491390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.491527 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.491619 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.491624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.492396 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.492399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.492960 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.492969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.493432 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.493436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.493524 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.493901 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.493913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.495121 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.496594 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.496598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.496600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.496774 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.497029 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.497032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.497034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.497110 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.497442 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.497445 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.497531 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.497621 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.497740 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260807 22:45:14.497906 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.491000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53000 (local address 127.26.189.253:37099)
0807 22:45:14.491136 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.491140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.491156 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:45:14.491257 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.491260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.491298 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.491356 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.491727 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.492304 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.493066 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.493335 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.493437 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.495246 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.495249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.495251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.495273 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.496490 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.496868 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.496871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.496874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.496950 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.497189 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.497192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.497194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.497355 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.497437 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.497674 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.497776 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:45:14.498600 16287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:45:14.498744 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.491129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.26.189.254:45861)
0807 22:45:14.491231 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.491235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.491247 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:45:14.491278 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.491280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.491312 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.491376 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.491971 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.492470 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.493198 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.493404 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.493492 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.495778 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.495781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.495783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.495803 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.497047 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.497415 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.497418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.497420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.497490 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.497731 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.497734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.497736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.497908 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.498142 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.498482 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.498609 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:45:14.506938 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.499254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45342)
0807 22:45:14.499343 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.499356 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.499420 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.499650 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.499653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.499800 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.499884 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.499890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.500782 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.500786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.501551 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.501560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.502089 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.502093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.502147 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.502532 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.502545 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.504064 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.505644 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.505647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.505649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.505823 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.506092 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.506094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.506095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.506169 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.506582 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.506584 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.506673 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.506753 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.506837 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 22:45:14.507366 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.499306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.26.189.252:41133)
0807 22:45:14.499508 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.499511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.499517 (+ 6us) server_negotiation.cc:407] Connection header received
0807 22:45:14.499535 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.499536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.499561 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.499605 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.499978 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.500688 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.501655 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.501999 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.502096 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.504188 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.504192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.504195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.504221 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.505528 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.505918 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.505921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.505922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.506001 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.506241 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.506244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.506246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.506497 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.506597 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.507130 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.507235 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260807 22:45:14.507668 16177 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } attempt: 3
I20260807 22:45:14.508788 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.499792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.26.189.253:37099)
0807 22:45:14.499885 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.499889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.499901 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:45:14.500025 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.500028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.500064 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.500124 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.500734 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.501469 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.502375 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.502686 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.502788 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.505247 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.505251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.505253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.505278 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.506695 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.507167 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.507170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.507172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.507250 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.507547 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.507550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.507552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.507744 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.507970 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.508558 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.508654 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:45:14.510867 16287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: Bootstrap starting.
I20260807 22:45:14.511998 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:45:14.511950 16075 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } attempt: 3
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:45:14.513713 16287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:14.514635 16287 log.cc:824] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:14.517275 16287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: No bootstrap required, opened a new log
Aug 07 22:45:14 dist-test-slave-4tdj krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142714, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:45:14.518071 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.508027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45352)
0807 22:45:14.508264 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.508277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.508341 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.508545 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.508548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.508712 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.508790 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.508797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.509743 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.509746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.510483 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.510494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.511016 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.511020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.511100 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.511495 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.511514 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.513179 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.515762 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.515765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.515768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.516013 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.517061 (+ 1048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.517064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.517066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.517163 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.517628 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.517632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.517724 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.517805 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.517924 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260807 22:45:14.518657 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.508231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.26.189.252:41133)
0807 22:45:14.508321 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.508325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.508336 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:45:14.508409 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.508412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.508448 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.508522 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.509089 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.509632 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.510603 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.510883 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.511294 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.513327 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.513331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.513333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.513357 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.515654 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.516636 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.516639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.516640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.516713 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.517279 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.517281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.517283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.517499 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.518335 (+ 836us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.518407 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.518507 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:45:14.521993 16177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f: Bootstrap starting.
I20260807 22:45:14.522202 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.512271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:41133 (local address 127.0.0.1:45360)
0807 22:45:14.512494 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:14.512507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:14.512586 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:14.513277 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:14.513280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:14.513457 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:14.513810 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.513823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.515043 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.515046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.515687 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:14.515698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.516546 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.516549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:14.516642 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.517021 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:14.517034 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:14.518647 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:14.520710 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.520713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.520715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.520894 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.521190 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:14.521193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:14.521196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:14.521284 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:14.521707 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:14.521710 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:14.521803 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 22:45:14.521901 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:45:14.522039 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 22:45:14.522682 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.512738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.26.189.252:41133)
0807 22:45:14.513048 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.513056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.513070 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:45:14.513099 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.513101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.513139 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.513931 (+ 792us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.514021 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.514940 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.516123 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.516445 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.516538 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.518968 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.518972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.518979 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.519002 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.520631 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.520980 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.520983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.520985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.521080 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.521361 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.521364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.521366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.521566 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.521660 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.522415 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.522525 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260807 22:45:14.524683 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.514334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.26.189.254:45861)
0807 22:45:14.514436 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.514440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.514693 (+ 253us) server_negotiation.cc:407] Connection header received
0807 22:45:14.514814 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.514818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.514861 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.514927 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.516471 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.517131 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.518360 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.518679 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.518742 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.520344 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.520347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.520350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.520375 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.522076 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.523317 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.523320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.523322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.523406 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.523731 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.523734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.523736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.523955 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.524098 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.524438 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.524541 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:45:14.524696 16177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:14.525569 16177 log.cc:824] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:14.527276 16075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: Bootstrap starting.
I20260807 22:45:14.528031 16287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.528277 16287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:14.528334 16287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c0c772ee9940efa9765981c34b3b3b, State: Initialized, Role: FOLLOWER
I20260807 22:45:14.528781 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.515099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.26.189.252:41133)
0807 22:45:14.519683 (+ 4584us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.519687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.519699 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:45:14.519729 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.519731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.519767 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.520055 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.521017 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.521893 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.522910 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.523220 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.523305 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.524712 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.524715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.524717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.524926 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.526846 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.527307 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.527311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.527313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.527396 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.527658 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.527661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.527662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.527850 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.527950 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.528542 (+ 592us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.528640 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4531,"thread_start_us":70,"threads_started":1}
I20260807 22:45:14.528806 16287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.529299 16311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.529418 16311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.530047 16075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:14.530128 16287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:14.530947 16075 log.cc:824] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:14.532055 16177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f: No bootstrap required, opened a new log
I20260807 22:45:14.533454 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:14.515456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.26.189.253:37099)
0807 22:45:14.515552 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:14.515556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:14.525254 (+ 9698us) server_negotiation.cc:407] Connection header received
0807 22:45:14.525345 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:14.525347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:14.525387 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:14.525450 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:14.526075 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.526835 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.527859 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:14.528151 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:14.528219 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:14.530725 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:14.530729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:14.530732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:14.530756 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:14.532075 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.532490 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.532492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.532493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.532559 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:14.532804 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:14.532806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:14.532807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:14.532952 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:14.533036 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:14.533263 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 22:45:14.533346 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:45:14.535503 16177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.535688 16177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:14.535746 16177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3089c8968b6b4427a10be299fef74a7f, State: Initialized, Role: FOLLOWER
I20260807 22:45:14.536091 16177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.536489 16313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.536644 16313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.537285 16177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:14.541565 16075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: No bootstrap required, opened a new log
W20260807 22:45:14.543128 16344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:45:14.543207 16344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:45:14.540545 16086 master_runner.cc:186] Error getting master registration for 127.26.189.254:45861: OK, instance_id { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" instance_seqno: 1786142714051515 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:45:14.543927 16086 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:45:14.543329 16185 master_runner.cc:186] Error getting master registration for 127.26.189.254:45861: OK, instance_id { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" instance_seqno: 1786142714051515 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:45:14.545648 16185 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 22:45:14.548421 16335 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:45:14.548501 16335 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:45:14.548753 16075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.548981 16075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:14.549038 16075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301b7179dab74b4ca7c410aa4596f91b, State: Initialized, Role: FOLLOWER
I20260807 22:45:14.549557 16075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.550083 16349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.550284 16349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.550999 16075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:45:14.557163 16370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:45:14.557312 16370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:45:14.561125 15968 master_runner.cc:263] No leader master found from master 301b7179dab74b4ca7c410aa4596f91b
I20260807 22:45:14.561192 15968 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:45:14.607820 16313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:14.608048 16313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.609190 16313 leader_election.cc:288] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301b7179dab74b4ca7c410aa4596f91b (127.26.189.254:45861), 39c0c772ee9940efa9765981c34b3b3b (127.26.189.252:41133)
I20260807 22:45:14.610128 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3089c8968b6b4427a10be299fef74a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c0c772ee9940efa9765981c34b3b3b" is_pre_election: true
I20260807 22:45:14.610535 16247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3089c8968b6b4427a10be299fef74a7f in term 0.
I20260807 22:45:14.610997 16109 leader_election.cc:302] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3089c8968b6b4427a10be299fef74a7f, 39c0c772ee9940efa9765981c34b3b3b; no voters:
I20260807 22:45:14.611020 16026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3089c8968b6b4427a10be299fef74a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301b7179dab74b4ca7c410aa4596f91b" is_pre_election: true
I20260807 22:45:14.611346 16026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3089c8968b6b4427a10be299fef74a7f in term 0.
I20260807 22:45:14.613643 16313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:14.613746 16313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:14.613804 16313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:14.615223 16313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.615844 16026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3089c8968b6b4427a10be299fef74a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301b7179dab74b4ca7c410aa4596f91b"
I20260807 22:45:14.615975 16026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:14.617303 16026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3089c8968b6b4427a10be299fef74a7f in term 1.
I20260807 22:45:14.617669 16112 leader_election.cc:302] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301b7179dab74b4ca7c410aa4596f91b, 3089c8968b6b4427a10be299fef74a7f; no voters:
I20260807 22:45:14.618041 16313 leader_election.cc:288] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [CANDIDATE]: Term 1 election: Requested vote from peers 301b7179dab74b4ca7c410aa4596f91b (127.26.189.254:45861), 39c0c772ee9940efa9765981c34b3b3b (127.26.189.252:41133)
I20260807 22:45:14.618151 16313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:14.618232 16313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [term 1 LEADER]: Becoming Leader. State: Replica: 3089c8968b6b4427a10be299fef74a7f, State: Running, Role: LEADER
I20260807 22:45:14.618578 16313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } }
I20260807 22:45:14.619611 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3089c8968b6b4427a10be299fef74a7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c0c772ee9940efa9765981c34b3b3b"
I20260807 22:45:14.619743 16247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:14.620980 16247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3089c8968b6b4427a10be299fef74a7f in term 1.
I20260807 22:45:14.622579 16313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3089c8968b6b4427a10be299fef74a7f. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.622732 16313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:14.623128 16376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:14.626540 16376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:14.633654 16247 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [term 1 FOLLOWER]: Refusing update from remote peer 3089c8968b6b4427a10be299fef74a7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:14.634306 16374 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:14.638005 16311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3089c8968b6b4427a10be299fef74a7f. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.638135 16311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.639068 16026 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [term 1 FOLLOWER]: Refusing update from remote peer 3089c8968b6b4427a10be299fef74a7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:14.639596 16374 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:14.642954 16349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3089c8968b6b4427a10be299fef74a7f. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.643069 16349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.652886 16374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.653038 16374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:14.654348 16311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.654469 16311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.654850 16379 mvcc.cc:205] Tried to move back new op lower bound from 7316040559132520448 to 7316040559089287168. Current Snapshot: MvccSnapshot[applied={T|T < 7316040559132520448}]
I20260807 22:45:14.655758 16313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.655870 16313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:14.656715 16349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3089c8968b6b4427a10be299fef74a7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301b7179dab74b4ca7c410aa4596f91b" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 45861 } } peers { permanent_uuid: "3089c8968b6b4427a10be299fef74a7f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 37099 } } peers { permanent_uuid: "39c0c772ee9940efa9765981c34b3b3b" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 41133 } } }
I20260807 22:45:14.656833 16349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:45:14.657786 16380 mvcc.cc:205] Tried to move back new op lower bound from 7316040559132520448 to 7316040559089287168. Current Snapshot: MvccSnapshot[applied={T|T < 7316040559132520448}]
I20260807 22:45:14.658362 16382 mvcc.cc:205] Tried to move back new op lower bound from 7316040559132520448 to 7316040559089287168. Current Snapshot: MvccSnapshot[applied={T|T < 7316040559132520448}]
I20260807 22:45:14.664016 16376 catalog_manager.cc:1347] Generated new cluster ID: c432e883f0c2420aad1bc1c5bed48474
I20260807 22:45:14.664103 16376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:14.682399 16376 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:45:14.683696 16376 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:14.702958 16376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3089c8968b6b4427a10be299fef74a7f: Generated new TSK 0
I20260807 22:45:14.703766 16376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:45:15.544492 16344 catalog_manager.cc:1259] Loaded cluster ID: c432e883f0c2420aad1bc1c5bed48474
I20260807 22:45:15.544579 16344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: loading cluster ID for follower catalog manager: success
I20260807 22:45:15.546253 16344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: acquiring CA information for follower catalog manager: success
I20260807 22:45:15.547457 16344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 39c0c772ee9940efa9765981c34b3b3b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:45:15.558586 16370 catalog_manager.cc:1259] Loaded cluster ID: c432e883f0c2420aad1bc1c5bed48474
I20260807 22:45:15.558653 16370 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: loading cluster ID for follower catalog manager: success
I20260807 22:45:15.560197 16370 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: acquiring CA information for follower catalog manager: success
I20260807 22:45:15.561266 16370 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 301b7179dab74b4ca7c410aa4596f91b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:45:15.771037 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 15968
I20260807 22:45:15.784480 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 16086
I20260807 22:45:15.794034 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 16185
2026-08-07T22:45:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3174 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:45:15 dist-test-slave-4tdj krb5kdc[16396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:45:15 dist-test-slave-4tdj krb5kdc[16396](info): set up 2 sockets
Aug 07 22:45:15 dist-test-slave-4tdj krb5kdc[16396](info): commencing operation
krb5kdc: starting...
W20260807 22:45:16.935652 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:45:17 dist-test-slave-4tdj krb5kdc[16396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:45:17.059010 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38397
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:17.166951 16414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:17.167189 16414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:17.167230 16414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:17.171109 16414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:17.171159 16414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:17.171177 16414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:17.171205 16414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:17.171231 16414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:17.176309 16414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:17.177260 16414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:17.178206 16414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:17.186141 16421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:17.186143 16425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:17.186362 16423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:17.186596 16414 server_base.cc:1034] running on GCE node
I20260807 22:45:17.187124 16414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:17.188151 16414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:17.189311 16414 hybrid_clock.cc:648] HybridClock initialized: now 1786142717189297 us; error 25 us; skew 500 ppm
Aug 07 22:45:17 dist-test-slave-4tdj krb5kdc[16396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142717, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:17.191891 16414 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:17.192698 16414 webserver.cc:460] Webserver started at http://127.26.189.254:44989/ using document root <none> and password file <none>
I20260807 22:45:17.193207 16414 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:17.193271 16414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:17.193459 16414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:17.195405 16414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "09118c518e7b419e990aeb34331d5f63"
format_stamp: "Formatted at 2026-08-07 22:45:17 on dist-test-slave-4tdj"
server_key: "74ec03a887331d468223fc9a63e2cc73"
server_key_iv: "0e6bd1e0cb86420eea613d577ad4fbd0"
server_key_version: "encryptionkey@0"
I20260807 22:45:17.195928 16414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "09118c518e7b419e990aeb34331d5f63"
format_stamp: "Formatted at 2026-08-07 22:45:17 on dist-test-slave-4tdj"
server_key: "74ec03a887331d468223fc9a63e2cc73"
server_key_iv: "0e6bd1e0cb86420eea613d577ad4fbd0"
server_key_version: "encryptionkey@0"
I20260807 22:45:17.199393 16414 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 22:45:17.201625 16439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:17.202781 16414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:45:17.202893 16414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "09118c518e7b419e990aeb34331d5f63"
format_stamp: "Formatted at 2026-08-07 22:45:17 on dist-test-slave-4tdj"
server_key: "74ec03a887331d468223fc9a63e2cc73"
server_key_iv: "0e6bd1e0cb86420eea613d577ad4fbd0"
server_key_version: "encryptionkey@0"
I20260807 22:45:17.202986 16414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:17.232792 16414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:17.235891 16414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:17.236068 16414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:17.242872 16414 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38397
I20260807 22:45:17.242892 16516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38397 every 8 connection(s)
I20260807 22:45:17.243763 16414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:17.245076 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 16414
I20260807 22:45:17.245215 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:45:17.245532 27383 external_mini_cluster.cc:1428] Setting key 5ec62982ad19376ca809d6b049c8e659
I20260807 22:45:17.246920 16517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:17.253021 16517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63: Bootstrap starting.
Aug 07 22:45:17 dist-test-slave-4tdj krb5kdc[16396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:17.255271 16517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:17.255942 16517 log.cc:824] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:17.257714 16517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63: No bootstrap required, opened a new log
I20260807 22:45:17.258810 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:17.246810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.26.189.254:38397)
0807 22:45:17.247202 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:17.247218 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:17.247253 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:45:17.247871 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:17.247886 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:17.248217 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:17.248482 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:17.249320 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:17.250287 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:17.251430 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:17.251866 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:17.251973 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:17.254555 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:17.254560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:17.254572 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:17.254600 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:17.256147 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:17.256542 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:17.256547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:17.256552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:17.256659 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:17.256906 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:17.256909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:17.256911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:17.257365 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:17.257537 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:17.257818 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 22:45:17.258085 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":129,"threads_started":1}
I20260807 22:45:17.260900 16517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09118c518e7b419e990aeb34331d5f63" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38397 } }
I20260807 22:45:17.261132 16517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:17.261195 16517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09118c518e7b419e990aeb34331d5f63, State: Initialized, Role: FOLLOWER
I20260807 22:45:17.261622 16517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09118c518e7b419e990aeb34331d5f63" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38397 } }
I20260807 22:45:17.261781 16517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:17.261829 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:45:17.261857 16517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:17.261950 16517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:17.263022 16517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09118c518e7b419e990aeb34331d5f63" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38397 } }
I20260807 22:45:17.263339 16517 leader_election.cc:302] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09118c518e7b419e990aeb34331d5f63; no voters:
I20260807 22:45:17.263675 16517 leader_election.cc:288] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:45:17.263734 16523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:17.263954 16523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [term 1 LEADER]: Becoming Leader. State: Replica: 09118c518e7b419e990aeb34331d5f63, State: Running, Role: LEADER
I20260807 22:45:17.264266 16523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09118c518e7b419e990aeb34331d5f63" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38397 } }
I20260807 22:45:17.264740 16517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:17.265898 16526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09118c518e7b419e990aeb34331d5f63. Latest consensus state: current_term: 1 leader_uuid: "09118c518e7b419e990aeb34331d5f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09118c518e7b419e990aeb34331d5f63" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38397 } } }
I20260807 22:45:17.265902 16525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09118c518e7b419e990aeb34331d5f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09118c518e7b419e990aeb34331d5f63" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38397 } } }
I20260807 22:45:17.266031 16525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:17.266031 16526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:17.266477 16537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:17.269410 16537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:17.274268 16537 catalog_manager.cc:1347] Generated new cluster ID: 9d8749105e7f4255840325de37c83a75
I20260807 22:45:17.274369 16537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:17.284832 16537 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:45:17.285748 16537 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:17.298578 16537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09118c518e7b419e990aeb34331d5f63: Generated new TSK 0
I20260807 22:45:17.299252 16537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:45:17.318717 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 16414
2026-08-07T22:45:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1528 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:45:17 dist-test-slave-4tdj krb5kdc[16554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:45:17 dist-test-slave-4tdj krb5kdc[16554](info): set up 2 sockets
Aug 07 22:45:17 dist-test-slave-4tdj krb5kdc[16554](info): commencing operation
krb5kdc: starting...
W20260807 22:45:18.450989 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:45:18 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:45:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:45:18.570992 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43995
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:18.679512 16577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:18.679745 16577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:18.679782 16577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:18.683640 16577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:18.683686 16577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:18.683704 16577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:18.683724 16577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:18.683738 16577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:18.688795 16577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:18.689711 16577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:18.690642 16577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:18.698509 16583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:18.698550 16586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:18.698515 16584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:18.698794 16577 server_base.cc:1034] running on GCE node
I20260807 22:45:18.699347 16577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:18.700361 16577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:18.701525 16577 hybrid_clock.cc:648] HybridClock initialized: now 1786142718701504 us; error 33 us; skew 500 ppm
Aug 07 22:45:18 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142718, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:18.712431 16577 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:18.713364 16577 webserver.cc:460] Webserver started at http://127.26.189.254:46071/ using document root <none> and password file <none>
I20260807 22:45:18.713891 16577 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:18.713959 16577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:18.714176 16577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:18.716256 16577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "abfbe055bf6d493e8604254395f48e7e"
format_stamp: "Formatted at 2026-08-07 22:45:18 on dist-test-slave-4tdj"
server_key: "4f998f9caf8c3374a3bff57195e91012"
server_key_iv: "d05469979e34b87300dea8ab99e7c65b"
server_key_version: "encryptionkey@0"
I20260807 22:45:18.716799 16577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "abfbe055bf6d493e8604254395f48e7e"
format_stamp: "Formatted at 2026-08-07 22:45:18 on dist-test-slave-4tdj"
server_key: "4f998f9caf8c3374a3bff57195e91012"
server_key_iv: "d05469979e34b87300dea8ab99e7c65b"
server_key_version: "encryptionkey@0"
I20260807 22:45:18.720388 16577 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:45:18.722643 16594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:18.723851 16577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:18.724000 16577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "abfbe055bf6d493e8604254395f48e7e"
format_stamp: "Formatted at 2026-08-07 22:45:18 on dist-test-slave-4tdj"
server_key: "4f998f9caf8c3374a3bff57195e91012"
server_key_iv: "d05469979e34b87300dea8ab99e7c65b"
server_key_version: "encryptionkey@0"
I20260807 22:45:18.724097 16577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:18.739308 16577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:18.742395 16577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:18.742574 16577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:18.749387 16577 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43995
I20260807 22:45:18.749639 16676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43995 every 8 connection(s)
I20260807 22:45:18.750360 16577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:18.752991 16677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:18.757354 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 16577
I20260807 22:45:18.757488 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:45:18.757799 27383 external_mini_cluster.cc:1428] Setting key 65b3a5b685a6195e8995df5bbfc33a38
I20260807 22:45:18.758530 16677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap starting.
I20260807 22:45:18.761183 16677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:18.761998 16677 log.cc:824] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:18.764029 16677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: No bootstrap required, opened a new log
Aug 07 22:45:18 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:18.767732 16677 raft_consensus.cc:348] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:18.767957 16677 raft_consensus.cc:374] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:18.768052 16677 raft_consensus.cc:729] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abfbe055bf6d493e8604254395f48e7e, State: Initialized, Role: FOLLOWER
I20260807 22:45:18.768499 16677 consensus_queue.cc:260] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:18.768659 16677 raft_consensus.cc:388] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:18.768731 16677 raft_consensus.cc:482] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:18.768826 16677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:18.769650 16677 raft_consensus.cc:504] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:18.769965 16677 leader_election.cc:302] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abfbe055bf6d493e8604254395f48e7e; no voters:
I20260807 22:45:18.770246 16677 leader_election.cc:288] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:45:18.770370 16682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:18.770594 16682 raft_consensus.cc:686] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 LEADER]: Becoming Leader. State: Replica: abfbe055bf6d493e8604254395f48e7e, State: Running, Role: LEADER
I20260807 22:45:18.770965 16682 consensus_queue.cc:237] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:18.771554 16677 sys_catalog.cc:564] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:18.772684 16684 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader abfbe055bf6d493e8604254395f48e7e. Latest consensus state: current_term: 1 leader_uuid: "abfbe055bf6d493e8604254395f48e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } } }
I20260807 22:45:18.772684 16683 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abfbe055bf6d493e8604254395f48e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } } }
I20260807 22:45:18.772819 16684 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:18.772819 16683 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:18.772904 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:18.759199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60680 (local address 127.26.189.254:43995)
0807 22:45:18.759650 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:18.759670 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:18.759714 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:45:18.760498 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:18.760522 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:18.760934 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:18.761293 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:18.761971 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:18.763324 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:18.764188 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:18.764652 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:18.764806 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:18.767389 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:18.767395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:18.767411 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:18.767441 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:18.769349 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:18.769801 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:18.769807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:18.769813 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:18.769925 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:18.770186 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:18.770189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:18.770191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:18.770761 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:18.771018 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:18.771621 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 22:45:18.771921 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
W20260807 22:45:18.776175 16699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:45:18.776242 16699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:45:18.776324 16694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:18.777002 16694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:18.782456 16694 catalog_manager.cc:1347] Generated new cluster ID: 6f2a3a8e12f4465c9dded8db2f36f6a0
I20260807 22:45:18.782510 16694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:18.808768 16694 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:45:18.809690 16694 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:18.814199 16694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Generated new TSK 0
I20260807 22:45:18.814962 16694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:45:18.850901 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43995
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:18.982120 16705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:18.982368 16705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:18.982427 16705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:18.986354 16705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:18.986405 16705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:18.986480 16705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:18.991412 16705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:18.992302 16705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:18.993228 16705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:45:19.002254 16705 server_base.cc:1034] running on GCE node
W20260807 22:45:19.002327 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:19.002321 16715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:19.002416 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:19.002868 16705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:19.003448 16705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:19.004611 16705 hybrid_clock.cc:648] HybridClock initialized: now 1786142719004589 us; error 30 us; skew 500 ppm
Aug 07 22:45:19 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142719, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:19.007292 16705 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:19.008085 16705 webserver.cc:460] Webserver started at http://127.26.189.193:44359/ using document root <none> and password file <none>
I20260807 22:45:19.008591 16705 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:19.008658 16705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:19.008849 16705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:19.010788 16705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "becdaa6bebe442ebba24dc07a19ce09c"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "a498820ef5e2d74eb023466e44ba49da"
server_key_iv: "d662729649f002223ea02a598c553c8d"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.011303 16705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "becdaa6bebe442ebba24dc07a19ce09c"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "a498820ef5e2d74eb023466e44ba49da"
server_key_iv: "d662729649f002223ea02a598c553c8d"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.014668 16705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:45:19.016845 16728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.017927 16705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:19.018030 16705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "becdaa6bebe442ebba24dc07a19ce09c"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "a498820ef5e2d74eb023466e44ba49da"
server_key_iv: "d662729649f002223ea02a598c553c8d"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.018126 16705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:19.041419 16705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:19.043924 16705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:19.044097 16705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:19.044636 16705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:19.045408 16705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:19.045454 16705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.045506 16705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:19.045532 16705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.054041 16705 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:34377
I20260807 22:45:19.054069 16901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:34377 every 8 connection(s)
I20260807 22:45:19.054879 16705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:19.057088 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 16705
I20260807 22:45:19.057225 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:45:19.057544 27383 external_mini_cluster.cc:1428] Setting key 8eb2a824dfc8fd649a096c446e9063f0
Aug 07 22:45:19 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142719, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:45:19.068818 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.056512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:52735 (local address 127.26.189.254:43995)
0807 22:45:19.056633 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.056636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.057477 (+ 841us) server_negotiation.cc:407] Connection header received
0807 22:45:19.058983 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.058987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.059038 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.059109 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:19.060229 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.060952 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.062530 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.062769 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.062841 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.065400 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:19.065405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:19.065408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:19.065438 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:19.066993 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.067453 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.067457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.067459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.067551 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.067849 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.067852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.067854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.068065 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:19.068168 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.068590 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.068698 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:45:19.069612 16907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.056830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:52735)
0807 22:45:19.057345 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.057386 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.058761 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.059268 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.059275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.059619 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:19.060060 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.060072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.061099 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.061103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.062415 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.062427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.062910 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.062913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.063098 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.063670 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:19.063692 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:19.065236 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:19.067103 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:19.067108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:19.067118 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:19.067357 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:19.067649 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:19.067651 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:19.067653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:19.067767 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:19.068149 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:19.068153 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:19.068386 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.068530 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.068773 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":150,"threads_started":1}
I20260807 22:45:19.070760 16902 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43995
I20260807 22:45:19.070973 16902 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:45:19.071468 16902 heartbeater.cc:511] Master 127.26.189.254:43995 requested a full tablet report, sending...
I20260807 22:45:19.073164 16616 ts_manager.cc:194] Registered new tserver with Master: becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377)
I20260807 22:45:19.074276 16616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:52735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:45:19.085034 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43995
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:19.191484 16910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:19.191721 16910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:19.191776 16910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:19.195659 16910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:19.195709 16910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:19.195783 16910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:19.200682 16910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:19.201581 16910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:19.202539 16910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:19.211443 16931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:19.211443 16922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:19.211447 16924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:19.211781 16910 server_base.cc:1034] running on GCE node
I20260807 22:45:19.212148 16910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:19.212711 16910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:19.213865 16910 hybrid_clock.cc:648] HybridClock initialized: now 1786142719213850 us; error 31 us; skew 500 ppm
Aug 07 22:45:19 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142719, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:19.216444 16910 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:19.217233 16910 webserver.cc:460] Webserver started at http://127.26.189.194:38119/ using document root <none> and password file <none>
I20260807 22:45:19.217737 16910 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:19.217801 16910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:19.217989 16910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:19.219956 16910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "e52de5c7df9bb5979fccb029fbbf397f"
server_key_iv: "8e25b0142e0c8b32a950d66081bcb3fe"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.220475 16910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "e52de5c7df9bb5979fccb029fbbf397f"
server_key_iv: "8e25b0142e0c8b32a950d66081bcb3fe"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.224467 16910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 22:45:19.227159 16940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.228402 16910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:19.228507 16910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "e52de5c7df9bb5979fccb029fbbf397f"
server_key_iv: "8e25b0142e0c8b32a950d66081bcb3fe"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.228615 16910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:19.242640 16910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:19.245446 16910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:19.245605 16910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:19.246129 16910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:19.246899 16910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:19.246943 16910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.246984 16910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:19.247013 16910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.255038 16910 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:43217
I20260807 22:45:19.255065 17106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:43217 every 8 connection(s)
I20260807 22:45:19.255831 16910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:19.260453 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 16910
I20260807 22:45:19.260591 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:45:19.260902 27383 external_mini_cluster.cc:1428] Setting key cf07cfedf5b19fbdb5e69a03d1951355
Aug 07 22:45:19 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142719, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:19.268923 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.257416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:39863 (local address 127.26.189.254:43995)
0807 22:45:19.257533 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.257537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.258276 (+ 739us) server_negotiation.cc:407] Connection header received
0807 22:45:19.259525 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.259528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.259576 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.259647 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:19.260871 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.261413 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.262750 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.263029 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.263122 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.265695 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:19.265698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:19.265700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:19.265723 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:19.267004 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.267549 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.267552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.267553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.267629 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.267957 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.267959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.267960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.268112 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:19.268179 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.268724 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.268812 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:45:19.270022 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.257721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:39863)
0807 22:45:19.258131 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.258171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.259348 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.259784 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.259790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.260100 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:19.260722 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.260745 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.261529 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.261533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.262638 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.262650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.263180 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.263184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.263370 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.263868 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:19.263887 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:19.265534 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:19.267126 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:19.267132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:19.267145 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:19.267467 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:19.267704 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:19.267707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:19.267709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:19.267880 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:19.268202 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:19.268208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:19.268498 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.268705 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.269021 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":101,"threads_started":1}
I20260807 22:45:19.271129 17107 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43995
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:45:19.271341 17107 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:19.271915 17107 heartbeater.cc:511] Master 127.26.189.254:43995 requested a full tablet report, sending...
I20260807 22:45:19.272939 16616 ts_manager.cc:194] Registered new tserver with Master: eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
I20260807 22:45:19.273553 16616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:39863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:45:19.288162 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43995
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:19.394307 17113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:19.394533 17113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:19.394573 17113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:19.398465 17113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:19.398517 17113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:19.398582 17113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:19.403627 17113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:19.404548 17113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:19.405501 17113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:19.414523 17125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:19.414529 17122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:19.414629 17113 server_base.cc:1034] running on GCE node
W20260807 22:45:19.414523 17129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:19.415158 17113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:19.415755 17113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:19.416908 17113 hybrid_clock.cc:648] HybridClock initialized: now 1786142719416890 us; error 31 us; skew 500 ppm
Aug 07 22:45:19 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142719, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:19.419368 17113 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:19.420161 17113 webserver.cc:460] Webserver started at http://127.26.189.195:38213/ using document root <none> and password file <none>
I20260807 22:45:19.420656 17113 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:19.420722 17113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:19.420918 17113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:19.422928 17113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "cb89d319b0cd4afaa08bbdd5915446ca"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "669e960fe0b44d5f3280dfade316791e"
server_key_iv: "678650151b10310a73ef536ff7295716"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.423456 17113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "cb89d319b0cd4afaa08bbdd5915446ca"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "669e960fe0b44d5f3280dfade316791e"
server_key_iv: "678650151b10310a73ef536ff7295716"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.426994 17113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:45:19.429340 17144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.430580 17113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:19.430685 17113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "cb89d319b0cd4afaa08bbdd5915446ca"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "669e960fe0b44d5f3280dfade316791e"
server_key_iv: "678650151b10310a73ef536ff7295716"
server_key_version: "encryptionkey@0"
I20260807 22:45:19.430776 17113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:19.445982 17113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:19.448559 17113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:19.448732 17113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:19.449275 17113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:19.450026 17113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:19.450074 17113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.450133 17113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:19.450157 17113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:19.458484 17113 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:39785
I20260807 22:45:19.458518 17303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:39785 every 8 connection(s)
I20260807 22:45:19.459301 17113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:19.463300 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 17113
I20260807 22:45:19.463414 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:45:19.463730 27383 external_mini_cluster.cc:1428] Setting key 4cb4bc25ca9e677518aaf587c93c5334
Aug 07 22:45:19 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142719, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:19.471974 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.460995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:48919 (local address 127.26.189.254:43995)
0807 22:45:19.461145 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.461150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.461951 (+ 801us) server_negotiation.cc:407] Connection header received
0807 22:45:19.463313 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.463317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.463377 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.463458 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:19.464729 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.465460 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.466667 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.466879 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.466960 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.469078 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:19.469082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:19.469083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:19.469107 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:19.470391 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.470778 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.470780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.470782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.470850 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.471085 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.471086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.471088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.471239 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:19.471347 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.471768 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.471861 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:45:19.472718 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.461370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:48919)
0807 22:45:19.461805 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.461847 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.463095 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.463682 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.463690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.464131 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:19.464581 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.464596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.465580 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.465583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.466549 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.466559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.466990 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.466992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.467133 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.467598 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:19.467613 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:19.468936 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:19.470471 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:19.470475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:19.470486 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:19.470706 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:19.470910 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:19.470912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:19.470913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:19.471024 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:19.471363 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:19.471368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:19.471573 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.471729 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.471949 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260807 22:45:19.473747 17304 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43995
I20260807 22:45:19.473932 17304 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:19.474376 17304 heartbeater.cc:511] Master 127.26.189.254:43995 requested a full tablet report, sending...
I20260807 22:45:19.475275 16616 ts_manager.cc:194] Registered new tserver with Master: cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785)
I20260807 22:45:19.475796 16616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:48919
I20260807 22:45:19.477375 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:19.485296 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.479340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.26.189.254:43995)
0807 22:45:19.479444 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.479447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.479622 (+ 175us) server_negotiation.cc:407] Connection header received
0807 22:45:19.479716 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.479718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.479751 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.479809 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:19.480384 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.480853 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.481585 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.481784 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.481848 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.482932 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:19.482935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:19.482937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:19.482957 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:19.484134 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.484432 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.484434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.484435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.484496 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:19.484673 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:19.484675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:19.484676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:19.484855 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:19.484949 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.485091 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.485179 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:45:19.487646 16616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:45:19.490473 16616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:45:19.503392 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.498090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39158 (local address 127.26.189.194:43217)
0807 22:45:19.498852 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.498864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.498889 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:45:19.498964 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.498970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.499178 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.499383 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:19.499643 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.500707 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.502148 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.502775 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.502887 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.503013 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.503168 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.503255 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":108,"threads_started":1}
I20260807 22:45:19.503381 17319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.497946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:43217 (local address 127.0.0.1:39158)
0807 22:45:19.498640 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.498688 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.498823 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.499374 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.499380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.499405 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:19.499526 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.499532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.500832 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.500835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.502029 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.502038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.502885 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.502889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.502993 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.503028 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.503131 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.503248 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":37,"thread_start_us":76,"threads_started":1}
I20260807 22:45:19.503743 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.498083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:39785 (local address 127.0.0.1:36532)
0807 22:45:19.499037 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.499049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.499150 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.499778 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.499780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.499798 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:19.499880 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.499886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.500959 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.500963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.502119 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.502129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.503376 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.503379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.503465 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.503488 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.503554 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.503615 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"mutex_wait_us":34,"thread_start_us":109,"threads_started":1}
I20260807 22:45:19.504251 17323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.498372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.26.189.195:39785)
0807 22:45:19.499373 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.499385 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.499406 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:45:19.499469 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.499474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.499659 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.499789 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:19.499988 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.500815 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.502265 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.503079 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.503194 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.503312 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.504013 (+ 701us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.504103 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":158,"threads_started":1}
I20260807 22:45:19.504496 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.498760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34377 (local address 127.0.0.1:46038)
0807 22:45:19.499613 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.499626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.499723 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.500372 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.500375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.500394 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:19.500477 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.500483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.501792 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.501796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.502893 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.502905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.504132 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.504135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.504220 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.504242 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.504320 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.504385 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":82,"threads_started":1}
I20260807 22:45:19.504930 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.499158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.26.189.193:34377)
0807 22:45:19.499961 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.499972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.499992 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:45:19.500061 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.500067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.500236 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.500381 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:19.500577 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.501637 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.503037 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.503839 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.503954 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.504070 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.504705 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.504798 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":140,"threads_started":1}
I20260807 22:45:19.506348 17008 tablet_service.cc:1467] Processing CreateTablet for tablet 54a8a481855348d6b34a5472e24408c4 (DEFAULT_TABLE table=test-table [id=b5832eb1312f4706a1153bad4ab4c909]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:19.506795 17212 tablet_service.cc:1467] Processing CreateTablet for tablet 54a8a481855348d6b34a5472e24408c4 (DEFAULT_TABLE table=test-table [id=b5832eb1312f4706a1153bad4ab4c909]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:19.507458 17008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a8a481855348d6b34a5472e24408c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:19.507666 17212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a8a481855348d6b34a5472e24408c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:19.507598 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 54a8a481855348d6b34a5472e24408c4 (DEFAULT_TABLE table=test-table [id=b5832eb1312f4706a1153bad4ab4c909]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:19.508510 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a8a481855348d6b34a5472e24408c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:19.513387 17328 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Bootstrap starting.
I20260807 22:45:19.515168 17330 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Bootstrap starting.
I20260807 22:45:19.516160 17328 tablet_bootstrap.cc:654] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:19.516914 17328 log.cc:824] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:19.517171 17330 tablet_bootstrap.cc:654] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:19.517553 17332 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap starting.
I20260807 22:45:19.517890 17330 log.cc:824] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:19.518872 17328 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: No bootstrap required, opened a new log
I20260807 22:45:19.519057 17328 ts_tablet_manager.cc:1397] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:45:19.519332 17332 tablet_bootstrap.cc:654] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:19.519857 17330 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: No bootstrap required, opened a new log
I20260807 22:45:19.520093 17330 ts_tablet_manager.cc:1397] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:45:19.520493 17332 log.cc:824] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:19.521960 17332 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: No bootstrap required, opened a new log
I20260807 22:45:19.522161 17332 ts_tablet_manager.cc:1397] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 22:45:19.522861 17328 raft_consensus.cc:348] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.523227 17328 raft_consensus.cc:374] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:19.523295 17328 raft_consensus.cc:729] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb89d319b0cd4afaa08bbdd5915446ca, State: Initialized, Role: FOLLOWER
I20260807 22:45:19.523759 17328 consensus_queue.cc:260] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.523712 17330 raft_consensus.cc:348] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.523929 17330 raft_consensus.cc:374] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:19.524019 17330 raft_consensus.cc:729] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb710f7bdb5e4fe3b55748d3cb3e9e38, State: Initialized, Role: FOLLOWER
I20260807 22:45:19.524405 17304 heartbeater.cc:503] Master 127.26.189.254:43995 was elected leader, sending a full tablet report...
I20260807 22:45:19.524559 17330 consensus_queue.cc:260] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.524739 17328 ts_tablet_manager.cc:1428] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:45:19.525413 17330 ts_tablet_manager.cc:1428] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260807 22:45:19.525708 17107 heartbeater.cc:503] Master 127.26.189.254:43995 was elected leader, sending a full tablet report...
I20260807 22:45:19.525534 17332 raft_consensus.cc:348] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.525781 17332 raft_consensus.cc:374] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:19.525828 17332 raft_consensus.cc:729] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: becdaa6bebe442ebba24dc07a19ce09c, State: Initialized, Role: FOLLOWER
I20260807 22:45:19.526196 17332 consensus_queue.cc:260] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.526826 16902 heartbeater.cc:503] Master 127.26.189.254:43995 was elected leader, sending a full tablet report...
I20260807 22:45:19.527493 17332 ts_tablet_manager.cc:1428] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
W20260807 22:45:19.555922 16905 tablet.cc:2369] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:45:19.710379 17305 tablet.cc:2369] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:19.729452 17337 raft_consensus.cc:482] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:19.729645 17337 raft_consensus.cc:504] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.730760 17337 leader_election.cc:288] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217), becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377)
I20260807 22:45:19.734704 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.731112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:54759 (local address 127.26.189.194:43217)
0807 22:45:19.731243 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.731247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.731262 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:45:19.731312 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.731314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.731358 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.731419 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:19.731718 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.732409 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.733529 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.734135 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.734245 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.734325 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.734496 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.734584 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:45:19.734709 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.730966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:43217 (local address 127.26.189.195:54759)
0807 22:45:19.731120 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.731136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.731223 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.731432 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.731434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.731446 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:19.731597 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.731603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.732500 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.732503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.733381 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.733392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.734243 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.734247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.734381 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.734407 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.734484 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.734555 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 22:45:19.735356 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.730966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34377 (local address 127.26.189.195:45653)
0807 22:45:19.731816 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.731831 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:19.731923 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:19.732126 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:19.732127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:19.732140 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:19.732243 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.732249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.732985 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.732989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.734006 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:19.734017 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.734996 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.734999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:19.735094 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.735116 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:45:19.735193 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:45:19.735246 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":516,"thread_start_us":97,"threads_started":1}
I20260807 22:45:19.735428 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.731079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:45653 (local address 127.26.189.193:34377)
0807 22:45:19.731226 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.731232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.731910 (+ 678us) server_negotiation.cc:407] Connection header received
0807 22:45:19.731993 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.731996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.732047 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.732122 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:19.732350 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.732858 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.734133 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.734708 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.734824 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.734889 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.735230 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.735309 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:45:19.735448 17042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a8a481855348d6b34a5472e24408c4" candidate_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" is_pre_election: true
I20260807 22:45:19.735785 17042 raft_consensus.cc:2454] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb89d319b0cd4afaa08bbdd5915446ca in term 0.
I20260807 22:45:19.736060 16833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a8a481855348d6b34a5472e24408c4" candidate_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "becdaa6bebe442ebba24dc07a19ce09c" is_pre_election: true
I20260807 22:45:19.736282 17145 leader_election.cc:302] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb89d319b0cd4afaa08bbdd5915446ca, eb710f7bdb5e4fe3b55748d3cb3e9e38; no voters:
I20260807 22:45:19.736341 16833 raft_consensus.cc:2454] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb89d319b0cd4afaa08bbdd5915446ca in term 0.
I20260807 22:45:19.736563 17337 raft_consensus.cc:2781] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:19.736738 17337 raft_consensus.cc:482] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:19.736802 17337 raft_consensus.cc:3037] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:19.737819 17337 raft_consensus.cc:504] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.738137 17337 leader_election.cc:288] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [CANDIDATE]: Term 1 election: Requested vote from peers eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217), becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377)
I20260807 22:45:19.738492 16833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a8a481855348d6b34a5472e24408c4" candidate_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "becdaa6bebe442ebba24dc07a19ce09c"
I20260807 22:45:19.738515 17042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a8a481855348d6b34a5472e24408c4" candidate_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
I20260807 22:45:19.738605 16833 raft_consensus.cc:3037] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:19.738613 17042 raft_consensus.cc:3037] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:19.739583 16833 raft_consensus.cc:2454] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb89d319b0cd4afaa08bbdd5915446ca in term 1.
I20260807 22:45:19.739593 17042 raft_consensus.cc:2454] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb89d319b0cd4afaa08bbdd5915446ca in term 1.
I20260807 22:45:19.739841 17148 leader_election.cc:302] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c, cb89d319b0cd4afaa08bbdd5915446ca; no voters:
I20260807 22:45:19.740052 17337 raft_consensus.cc:2781] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:19.740236 17337 raft_consensus.cc:686] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 1 LEADER]: Becoming Leader. State: Replica: cb89d319b0cd4afaa08bbdd5915446ca, State: Running, Role: LEADER
I20260807 22:45:19.740561 17337 consensus_queue.cc:237] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:19.743583 16615 catalog_manager.cc:5495] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca reported cstate change: term changed from 0 to 1, leader changed from <none> to cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195). New cstate: current_term: 1 leader_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } health_report { overall_health: UNKNOWN } } }
W20260807 22:45:19.756856 17109 tablet.cc:2369] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:19.793390 17323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.790378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.26.189.195:39785)
0807 22:45:19.790501 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:19.790505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:19.790518 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:45:19.790594 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:19.790596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:19.790651 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:19.790737 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:19.790933 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.791436 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.792200 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:19.792403 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:19.792470 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:19.792713 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:19.793107 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:19.793201 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:19.793229 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 22:45:19.793279 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:45:19.798892 17042 raft_consensus.cc:1261] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Refusing update from remote peer cb89d319b0cd4afaa08bbdd5915446ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:19.798890 16833 raft_consensus.cc:1261] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Refusing update from remote peer cb89d319b0cd4afaa08bbdd5915446ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:19.799623 17343 consensus_queue.cc:1035] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:19.799795 17337 consensus_queue.cc:1035] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:19.807624 17347 mvcc.cc:205] Tried to move back new op lower bound from 7316040580289548288 to 7316040580059889664. Current Snapshot: MvccSnapshot[applied={T|T < 7316040580289548288}]
I20260807 22:45:19.809864 17348 mvcc.cc:205] Tried to move back new op lower bound from 7316040580289548288 to 7316040580059889664. Current Snapshot: MvccSnapshot[applied={T|T < 7316040580289548288}]
I20260807 22:45:19.814755 16615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:19.814921 16615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:19.817221 16615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Sending DeleteTablet for 3 replicas of tablet 54a8a481855348d6b34a5472e24408c4
I20260807 22:45:19.817971 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 16705
I20260807 22:45:19.817981 17212 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a8a481855348d6b34a5472e24408c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:19 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:36532
I20260807 22:45:19.818146 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a8a481855348d6b34a5472e24408c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:19 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:39158
I20260807 22:45:19.828212 17364 tablet_replica.cc:330] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: stopping tablet replica
I20260807 22:45:19.828549 17364 raft_consensus.cc:2229] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:19.828895 17364 raft_consensus.cc:2258] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:19.829824 17361 tablet_replica.cc:330] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: stopping tablet replica
I20260807 22:45:19.830170 17361 raft_consensus.cc:2229] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:45:19.830539 17364 ts_tablet_manager.cc:1905] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:19.830583 17361 raft_consensus.cc:2258] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:19.832099 17361 ts_tablet_manager.cc:1905] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 22:45:19.832974 16599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:34377 (error 108)
I20260807 22:45:19.833528 17364 ts_tablet_manager.cc:1918] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:19.833629 17364 log.cc:1196] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/54a8a481855348d6b34a5472e24408c4
I20260807 22:45:19.833942 17364 ts_tablet_manager.cc:1939] T 54a8a481855348d6b34a5472e24408c4 P eb710f7bdb5e4fe3b55748d3cb3e9e38: Deleting consensus metadata
I20260807 22:45:19.833943 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 16910
I20260807 22:45:19.835086 17361 ts_tablet_manager.cc:1918] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:19.835274 17361 log.cc:1196] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/54a8a481855348d6b34a5472e24408c4
I20260807 22:45:19.835871 17361 ts_tablet_manager.cc:1939] T 54a8a481855348d6b34a5472e24408c4 P cb89d319b0cd4afaa08bbdd5915446ca: Deleting consensus metadata
I20260807 22:45:19.842741 16598 catalog_manager.cc:4841] TS cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): tablet 54a8a481855348d6b34a5472e24408c4 (table test-table [id=b5832eb1312f4706a1153bad4ab4c909]) successfully deleted
I20260807 22:45:19.844086 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.843760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34377 (local address 127.0.0.1:46040)
0807 22:45:19.843939 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.844011 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260807 22:45:19.844291 16599 catalog_manager.cc:4584] TS becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54a8a481855348d6b34a5472e24408c4: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34377: connect: Connection refused (error 111)
I20260807 22:45:19.850037 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 17113
I20260807 22:45:19.853478 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:19.850191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:43217 (local address 127.0.0.1:39170)
0807 22:45:19.853335 (+ 3144us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:19.853394 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.194:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3058}
I20260807 22:45:19.859124 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 16577
I20260807 22:45:19.872164 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43995
--webserver_interface=127.26.189.254
--webserver_port=46071
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:19.981613 17366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:19.981860 17366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:19.981916 17366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:19.985873 17366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:19.985926 17366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:19.985952 17366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:19.985975 17366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:19.985996 17366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:19.991176 17366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=46071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:19.992134 17366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:19.993091 17366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:45:20.006767 17366 server_base.cc:1034] running on GCE node
W20260807 22:45:20.006726 17379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.006711 17377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.006711 17383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.007552 17366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:20.008733 17366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:20.009900 17366 hybrid_clock.cc:648] HybridClock initialized: now 1786142720009881 us; error 30 us; skew 500 ppm
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:20.012676 17366 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:20.013566 17366 webserver.cc:460] Webserver started at http://127.26.189.254:46071/ using document root <none> and password file <none>
I20260807 22:45:20.014125 17366 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:20.014209 17366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:20.018045 17366 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:20.020200 17395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.021392 17366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 22:45:20.021518 17366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "abfbe055bf6d493e8604254395f48e7e"
format_stamp: "Formatted at 2026-08-07 22:45:18 on dist-test-slave-4tdj"
server_key: "4f998f9caf8c3374a3bff57195e91012"
server_key_iv: "d05469979e34b87300dea8ab99e7c65b"
server_key_version: "encryptionkey@0"
I20260807 22:45:20.022096 17366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:20.042047 17366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:20.045259 17366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:20.045460 17366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:20.052428 17366 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43995
I20260807 22:45:20.052687 17478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43995 every 8 connection(s)
I20260807 22:45:20.053515 17366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:20.057770 17479 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:45:20.058449 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 17366
I20260807 22:45:20.058970 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:34377
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:43995
--webserver_port=44359
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:20.059809 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap starting.
I20260807 22:45:20.077098 17479 log.cc:824] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:20.086458 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:20.086802 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap complete.
I20260807 22:45:20.090726 17479 raft_consensus.cc:348] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:20.091130 17479 raft_consensus.cc:729] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: abfbe055bf6d493e8604254395f48e7e, State: Initialized, Role: FOLLOWER
I20260807 22:45:20.091625 17479 consensus_queue.cc:260] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:20.091749 17479 raft_consensus.cc:388] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:20.091818 17479 raft_consensus.cc:482] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:20.091908 17479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:45:20.093747 17479 raft_consensus.cc:504] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:20.094056 17479 leader_election.cc:302] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abfbe055bf6d493e8604254395f48e7e; no voters:
I20260807 22:45:20.094398 17479 leader_election.cc:288] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:45:20.094478 17487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:45:20.094660 17487 raft_consensus.cc:686] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 LEADER]: Becoming Leader. State: Replica: abfbe055bf6d493e8604254395f48e7e, State: Running, Role: LEADER
I20260807 22:45:20.094940 17487 consensus_queue.cc:237] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:20.095630 17479 sys_catalog.cc:564] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:20.095860 17487 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader abfbe055bf6d493e8604254395f48e7e. Latest consensus state: current_term: 2 leader_uuid: "abfbe055bf6d493e8604254395f48e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } } }
I20260807 22:45:20.095966 17487 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:20.096331 17492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:20.096346 17489 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "abfbe055bf6d493e8604254395f48e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } } }
I20260807 22:45:20.096582 17489 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:20.100980 17492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:20.101606 17492 catalog_manager.cc:1259] Loaded cluster ID: 6f2a3a8e12f4465c9dded8db2f36f6a0
I20260807 22:45:20.101670 17492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:20.103086 17492 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 22:45:20.107841 17492 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 22:45:20.113134 17492 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:20.113852 17492 catalog_manager.cc:5879] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Loaded TSK: 0
I20260807 22:45:20.115043 17492 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:45:20.173842 17482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:20.174108 17482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:20.174168 17482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:20.178226 17482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:20.178287 17482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:20.178395 17482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:20.183538 17482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:34377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=44359
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:20.184508 17482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:20.185550 17482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:20.195675 17520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.195669 17519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.195879 17482 server_base.cc:1034] running on GCE node
W20260807 22:45:20.195981 17525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.196381 17482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:20.197005 17482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:20.198169 17482 hybrid_clock.cc:648] HybridClock initialized: now 1786142720198145 us; error 41 us; skew 500 ppm
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:20.200888 17482 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:20.201793 17482 webserver.cc:460] Webserver started at http://127.26.189.193:44359/ using document root <none> and password file <none>
I20260807 22:45:20.202364 17482 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:20.202474 17482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:20.206244 17482 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:20.208423 17537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.209698 17482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:20.209805 17482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "becdaa6bebe442ebba24dc07a19ce09c"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "a498820ef5e2d74eb023466e44ba49da"
server_key_iv: "d662729649f002223ea02a598c553c8d"
server_key_version: "encryptionkey@0"
I20260807 22:45:20.210378 17482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:20.231093 17482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:20.234086 17482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:20.234277 17482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:20.234894 17482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:20.235970 17544 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:45:20.239982 17482 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:45:20.240037 17482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260807 22:45:20.240087 17482 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:45:20.241686 17482 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:45:20.241743 17482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 22:45:20.242033 17544 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap starting.
I20260807 22:45:20.252158 17482 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:34377
I20260807 22:45:20.252218 17717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:34377 every 8 connection(s)
I20260807 22:45:20.253324 17482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:20.256826 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 17482
I20260807 22:45:20.257331 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:43217
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:43995
--webserver_port=38119
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:20.263391 17544 log.cc:824] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:20.270957 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.255459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:60819)
0807 22:45:20.255940 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.255981 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.256855 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.258378 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.258386 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.258845 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:20.259406 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.259421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.260746 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.260749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.261598 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.261609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.262341 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.262346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.262561 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.263184 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:20.263204 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:20.264997 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:20.267620 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:20.267626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:20.267636 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:20.267961 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:20.268362 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:20.268365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:20.268367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:20.268532 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:20.269239 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:20.269245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:20.269511 (+ 266us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.269706 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.270017 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260807 22:45:20.271173 17544 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:20.271476 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.255377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:60819 (local address 127.26.189.254:43995)
0807 22:45:20.255718 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.255739 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.256105 (+ 366us) server_negotiation.cc:407] Connection header received
0807 22:45:20.257355 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.257372 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.257862 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.258281 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:20.259576 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.260555 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.261720 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.262187 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.262367 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.265252 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:20.265260 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:20.265279 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:20.265314 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:20.267481 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.268085 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.268091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.268097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.268265 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.268631 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.268634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.268635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.269110 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:20.269269 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.269857 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.270173 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
I20260807 22:45:20.271565 17544 tablet_bootstrap.cc:492] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap complete.
I20260807 22:45:20.272612 17544 ts_tablet_manager.cc:1397] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.004s
I20260807 22:45:20.273800 17719 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43995
I20260807 22:45:20.274035 17719 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:20.274608 17719 heartbeater.cc:511] Master 127.26.189.254:43995 requested a full tablet report, sending...
I20260807 22:45:20.275229 17544 raft_consensus.cc:348] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.275619 17544 raft_consensus.cc:729] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: becdaa6bebe442ebba24dc07a19ce09c, State: Initialized, Role: FOLLOWER
I20260807 22:45:20.276117 17544 consensus_queue.cc:260] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.277187 17544 ts_tablet_manager.cc:1428] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260807 22:45:20.277355 17422 ts_manager.cc:194] Registered new tserver with Master: becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377)
I20260807 22:45:20.280602 17422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:60819
I20260807 22:45:20.283160 17719 heartbeater.cc:503] Master 127.26.189.254:43995 was elected leader, sending a full tablet report...
I20260807 22:45:20.286561 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.280273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.26.189.193:34377)
0807 22:45:20.282464 (+ 2191us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.282473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.282491 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:45:20.282540 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.282543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.282692 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.282823 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:20.283080 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.283975 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.285395 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.285908 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.286020 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.286152 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.286368 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.286457 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2113,"thread_start_us":80,"threads_started":1}
I20260807 22:45:20.286558 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.280203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34377 (local address 127.0.0.1:46044)
0807 22:45:20.280609 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.280653 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.280773 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.282834 (+ 2061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.282840 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.282863 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:20.282978 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.282984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.284107 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.284111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.285273 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.285284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.286020 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.286024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.286133 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.286170 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.286269 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.286421 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":126,"threads_started":1}
I20260807 22:45:20.287513 17613 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a8a481855348d6b34a5472e24408c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:19 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:46044
I20260807 22:45:20.288416 17734 tablet_replica.cc:330] stopping tablet replica
I20260807 22:45:20.288686 17734 raft_consensus.cc:2229] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:20.288897 17734 raft_consensus.cc:2258] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:20.289815 17734 ts_tablet_manager.cc:1905] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:20.293140 17734 ts_tablet_manager.cc:1918] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:20.293237 17734 log.cc:1196] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/54a8a481855348d6b34a5472e24408c4
I20260807 22:45:20.293509 17734 ts_tablet_manager.cc:1939] T 54a8a481855348d6b34a5472e24408c4 P becdaa6bebe442ebba24dc07a19ce09c: Deleting consensus metadata
I20260807 22:45:20.295118 17402 catalog_manager.cc:4841] TS becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377): tablet 54a8a481855348d6b34a5472e24408c4 (table test-table [id=b5832eb1312f4706a1153bad4ab4c909]) successfully deleted
W20260807 22:45:20.374869 17727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:20.375101 17727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:20.375157 17727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:20.379086 17727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:20.379136 17727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:20.379212 17727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:20.384199 17727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:43217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=38119
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:20.385142 17727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:20.386101 17727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:20.395340 17747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.395416 17727 server_base.cc:1034] running on GCE node
W20260807 22:45:20.395356 17750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.395341 17746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.396014 17727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:20.396591 17727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:20.397754 17727 hybrid_clock.cc:648] HybridClock initialized: now 1786142720397734 us; error 29 us; skew 500 ppm
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:20.400380 17727 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:20.401140 17727 webserver.cc:460] Webserver started at http://127.26.189.194:38119/ using document root <none> and password file <none>
I20260807 22:45:20.401638 17727 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:20.401722 17727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:20.405344 17727 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:20.407377 17759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.408510 17727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:20.408607 17727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "e52de5c7df9bb5979fccb029fbbf397f"
server_key_iv: "8e25b0142e0c8b32a950d66081bcb3fe"
server_key_version: "encryptionkey@0"
I20260807 22:45:20.409113 17727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:20.420996 17727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:20.423840 17727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:20.423996 17727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:20.424531 17727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:20.425338 17727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:20.425388 17727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.425431 17727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:20.425459 17727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.433588 17727 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:43217
I20260807 22:45:20.433656 17933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:43217 every 8 connection(s)
I20260807 22:45:20.434544 17727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:20.436354 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 17727
I20260807 22:45:20.436765 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:39785
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:43995
--webserver_port=38213
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:20.448323 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.436226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:40849 (local address 127.26.189.254:43995)
0807 22:45:20.436388 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.436394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.437348 (+ 954us) server_negotiation.cc:407] Connection header received
0807 22:45:20.438717 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.438721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.438784 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.438883 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:20.439994 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.440761 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.441850 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.442225 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.442364 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.444797 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:20.444801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:20.444804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:20.444835 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:20.446366 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.446825 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.446829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.446831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.446926 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.447240 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.447243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.447244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.447469 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:20.447568 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.448006 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.448151 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:45:20.449052 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.436608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:40849)
0807 22:45:20.437185 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.437235 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.438503 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.439028 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.439034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.439376 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:20.439846 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.439860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.440875 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.440877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.441732 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.441741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.442389 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.442392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.442533 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.443059 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:20.443075 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:20.444616 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:20.446471 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:20.446476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:20.446486 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:20.446731 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:20.447006 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:20.447008 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:20.447009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:20.447155 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:20.447563 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:20.447567 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:20.447788 (+ 221us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.447932 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.448164 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":108,"threads_started":1}
I20260807 22:45:20.450117 17936 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43995
I20260807 22:45:20.450371 17936 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:20.450821 17936 heartbeater.cc:511] Master 127.26.189.254:43995 requested a full tablet report, sending...
I20260807 22:45:20.451807 17422 ts_manager.cc:194] Registered new tserver with Master: eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
I20260807 22:45:20.452528 17422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:40849
W20260807 22:45:20.548441 17942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:20.548677 17942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:20.548717 17942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:20.552693 17942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:20.552747 17942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:20.552800 17942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:20.557791 17942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:39785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=38213
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:20.558746 17942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:20.559670 17942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:20.568822 17949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.568861 17950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.568857 17954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.568923 17942 server_base.cc:1034] running on GCE node
I20260807 22:45:20.569465 17942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:20.570060 17942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:20.571213 17942 hybrid_clock.cc:648] HybridClock initialized: now 1786142720571194 us; error 33 us; skew 500 ppm
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:20.573800 17942 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:20.574709 17942 webserver.cc:460] Webserver started at http://127.26.189.195:38213/ using document root <none> and password file <none>
I20260807 22:45:20.575205 17942 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:20.575275 17942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:20.578930 17942 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:20.580926 17970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.582077 17942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:20.582198 17942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "cb89d319b0cd4afaa08bbdd5915446ca"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "669e960fe0b44d5f3280dfade316791e"
server_key_iv: "678650151b10310a73ef536ff7295716"
server_key_version: "encryptionkey@0"
I20260807 22:45:20.582764 17942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:20.592644 17942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:20.595382 17942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:20.595610 17942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:20.596189 17942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:20.597039 17942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:20.597085 17942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.597139 17942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:20.597167 17942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.605536 17942 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:39785
I20260807 22:45:20.605556 18138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:39785 every 8 connection(s)
I20260807 22:45:20.606536 17942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:20.613941 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 17942
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:20.620267 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.608187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:41869 (local address 127.26.189.254:43995)
0807 22:45:20.608309 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.608312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.609113 (+ 801us) server_negotiation.cc:407] Connection header received
0807 22:45:20.610399 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.610402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.610451 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.610529 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:20.611675 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.612206 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.613387 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.613650 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.613759 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.616858 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:20.616863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:20.616866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:20.616894 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:20.618188 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.618726 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.618730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.618731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.618825 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.619136 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.619140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.619141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.619352 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:20.619450 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.620005 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.620151 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:45:20.621243 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.608533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:41869)
0807 22:45:20.608964 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.609006 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.610177 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.610703 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.610709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.611070 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:20.611524 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.611539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.612303 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.612306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.613267 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.613277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.613905 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.613908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.614058 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.614938 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:20.614959 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:20.616700 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:20.618321 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:20.618327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:20.618341 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:20.618634 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:20.618910 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:20.618912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:20.618913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:20.619041 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:20.619438 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:20.619443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:20.619722 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.619912 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.620232 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":101,"threads_started":1}
I20260807 22:45:20.622346 18139 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43995
I20260807 22:45:20.622571 18139 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:20.623016 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.614603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.26.189.254:43995)
0807 22:45:20.614823 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.614828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.615020 (+ 192us) server_negotiation.cc:407] Connection header received
0807 22:45:20.615171 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.615173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.615209 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.615279 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:20.616063 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.616809 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.617891 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.618111 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.618186 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.619578 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:20.619581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:20.619584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:20.619622 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:20.621135 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.621812 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.621815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.621816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.621886 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:20.622154 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:20.622157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:20.622159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:20.622389 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:20.622577 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.622756 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.622873 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260807 22:45:20.623133 18139 heartbeater.cc:511] Master 127.26.189.254:43995 requested a full tablet report, sending...
I20260807 22:45:20.624137 17422 ts_manager.cc:194] Registered new tserver with Master: cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785)
I20260807 22:45:20.624774 17422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:41869
I20260807 22:45:20.624856 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:20.628840 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.625457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.26.189.254:43995)
0807 22:45:20.625553 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.625558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.625749 (+ 191us) server_negotiation.cc:407] Connection header received
0807 22:45:20.625875 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.625878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.625961 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.626043 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:20.626258 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.626813 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.627618 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.627835 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.627916 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.628197 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:20.628515 (+ 318us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:20.628622 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.628684 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.628735 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:45:20.629381 17422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:45:20.632510 17422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:45:20.639740 17613 tablet_service.cc:1467] Processing CreateTablet for tablet 14fc478ae61749df89efd68a3094fd7d (DEFAULT_TABLE table=test-table [id=c784873fa20b41dda7dbc7664173c8a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:20.640390 17613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fc478ae61749df89efd68a3094fd7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:20.643002 17544 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap starting.
I20260807 22:45:20.643908 17544 tablet_bootstrap.cc:654] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:20.643986 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.638918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:43217 (local address 127.0.0.1:39176)
0807 22:45:20.639309 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.639322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.639401 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.640075 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.640078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.640106 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:20.640221 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.640228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.641417 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.641419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.642602 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.642609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.643584 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.643588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.643690 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.643712 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.643781 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.643853 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":38,"thread_start_us":66,"threads_started":1}
I20260807 22:45:20.644016 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.639292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39176 (local address 127.26.189.194:43217)
0807 22:45:20.639657 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.639668 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.639690 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:45:20.639762 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.639767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.639951 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.640097 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:20.640322 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.641300 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.642721 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.643475 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.643585 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.643725 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.643808 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.643893 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260807 22:45:20.644182 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.638693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:39785 (local address 127.0.0.1:36542)
0807 22:45:20.638802 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.638817 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.638893 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.639657 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.639660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.639680 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:20.639772 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.639778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.641346 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.641350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.642585 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.642598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.643693 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.643696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.643783 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.643808 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.643856 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.643932 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":18}
I20260807 22:45:20.644826 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.638844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.26.189.195:39785)
0807 22:45:20.639134 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.639145 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.639167 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:45:20.639242 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.639248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.639464 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.639677 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:20.639862 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.641230 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.642727 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.643558 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.644427 (+ 869us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.644548 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.644601 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.644691 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":115,"threads_started":1}
I20260807 22:45:20.645736 17544 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: No bootstrap required, opened a new log
I20260807 22:45:20.645818 17544 ts_tablet_manager.cc:1397] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:45:20.646359 17544 raft_consensus.cc:348] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.646466 17544 raft_consensus.cc:374] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:20.646508 17544 raft_consensus.cc:729] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: becdaa6bebe442ebba24dc07a19ce09c, State: Initialized, Role: FOLLOWER
I20260807 22:45:20.646623 17544 consensus_queue.cc:260] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.646869 17544 ts_tablet_manager.cc:1428] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:45:20.647125 17832 tablet_service.cc:1467] Processing CreateTablet for tablet 14fc478ae61749df89efd68a3094fd7d (DEFAULT_TABLE table=test-table [id=c784873fa20b41dda7dbc7664173c8a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:20.647573 18043 tablet_service.cc:1467] Processing CreateTablet for tablet 14fc478ae61749df89efd68a3094fd7d (DEFAULT_TABLE table=test-table [id=c784873fa20b41dda7dbc7664173c8a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:20.648245 17832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fc478ae61749df89efd68a3094fd7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:20.648600 18043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fc478ae61749df89efd68a3094fd7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:20.656342 18164 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Bootstrap starting.
I20260807 22:45:20.656807 18162 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Bootstrap starting.
I20260807 22:45:20.659550 18162 tablet_bootstrap.cc:654] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:20.659550 18164 tablet_bootstrap.cc:654] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:20.660491 18164 log.cc:824] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:20.660491 18162 log.cc:824] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:20.662688 18162 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: No bootstrap required, opened a new log
I20260807 22:45:20.662688 18164 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: No bootstrap required, opened a new log
I20260807 22:45:20.662889 18164 ts_tablet_manager.cc:1397] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 22:45:20.662889 18162 ts_tablet_manager.cc:1397] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:45:20.666792 18164 raft_consensus.cc:348] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.666805 18162 raft_consensus.cc:348] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.667064 18164 raft_consensus.cc:374] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:20.667064 18162 raft_consensus.cc:374] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:20.667130 18164 raft_consensus.cc:729] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb89d319b0cd4afaa08bbdd5915446ca, State: Initialized, Role: FOLLOWER
I20260807 22:45:20.667130 18162 raft_consensus.cc:729] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb710f7bdb5e4fe3b55748d3cb3e9e38, State: Initialized, Role: FOLLOWER
I20260807 22:45:20.667608 18162 consensus_queue.cc:260] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.667611 18164 consensus_queue.cc:260] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.668278 17936 heartbeater.cc:503] Master 127.26.189.254:43995 was elected leader, sending a full tablet report...
I20260807 22:45:20.668468 18139 heartbeater.cc:503] Master 127.26.189.254:43995 was elected leader, sending a full tablet report...
I20260807 22:45:20.668644 18164 ts_tablet_manager.cc:1428] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:45:20.668643 18162 ts_tablet_manager.cc:1428] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 22:45:20.685460 17939 tablet.cc:2369] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:20.740658 17730 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:20.740864 17730 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.741899 17730 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
I20260807 22:45:20.745652 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.742284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:43853 (local address 127.26.189.195:39785)
0807 22:45:20.742453 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.742457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.742601 (+ 144us) server_negotiation.cc:407] Connection header received
0807 22:45:20.742732 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.742735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.742780 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.742854 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:20.743137 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.743616 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.744690 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.745077 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.745193 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.745255 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.745465 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.745531 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:45:20.745659 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.742307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:48779 (local address 127.26.189.194:43217)
0807 22:45:20.742433 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.742437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.742454 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:45:20.742555 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.742558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.742616 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.742694 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:20.742973 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.743528 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.744605 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.745021 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.745136 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.745188 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.745463 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.745539 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:45:20.745674 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.742110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:43217 (local address 127.26.189.193:48779)
0807 22:45:20.742334 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.742350 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.742441 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.742712 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.742715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.742734 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:20.742866 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.742873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.743646 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.743650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.744455 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.744465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.745159 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.745163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.745286 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.745361 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.745425 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.745514 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":40}
I20260807 22:45:20.745698 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.742117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:39785 (local address 127.26.189.193:43853)
0807 22:45:20.742503 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.742516 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:20.742628 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:20.742872 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:20.742874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:20.742889 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:20.743027 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.743034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.743778 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.743781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.744565 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:20.744576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.745240 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.745244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:20.745335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.745371 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:45:20.745439 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 22:45:20.745539 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":63,"thread_start_us":90,"threads_started":1}
I20260807 22:45:20.746371 17863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc478ae61749df89efd68a3094fd7d" candidate_uuid: "becdaa6bebe442ebba24dc07a19ce09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" is_pre_election: true
I20260807 22:45:20.746379 18070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc478ae61749df89efd68a3094fd7d" candidate_uuid: "becdaa6bebe442ebba24dc07a19ce09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" is_pre_election: true
I20260807 22:45:20.746699 18070 raft_consensus.cc:2454] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate becdaa6bebe442ebba24dc07a19ce09c in term 0.
I20260807 22:45:20.746699 17863 raft_consensus.cc:2454] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate becdaa6bebe442ebba24dc07a19ce09c in term 0.
I20260807 22:45:20.747218 17540 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c, cb89d319b0cd4afaa08bbdd5915446ca; no voters:
I20260807 22:45:20.747483 17730 raft_consensus.cc:2781] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:20.747560 17730 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:20.747617 17730 raft_consensus.cc:3037] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:20.748665 17730 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.748974 17730 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 1 election: Requested vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
I20260807 22:45:20.749294 17863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc478ae61749df89efd68a3094fd7d" candidate_uuid: "becdaa6bebe442ebba24dc07a19ce09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
I20260807 22:45:20.749303 18070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc478ae61749df89efd68a3094fd7d" candidate_uuid: "becdaa6bebe442ebba24dc07a19ce09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb89d319b0cd4afaa08bbdd5915446ca"
I20260807 22:45:20.749408 17863 raft_consensus.cc:3037] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:20.749429 18070 raft_consensus.cc:3037] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:20.750449 18070 raft_consensus.cc:2454] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate becdaa6bebe442ebba24dc07a19ce09c in term 1.
I20260807 22:45:20.750470 17863 raft_consensus.cc:2454] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate becdaa6bebe442ebba24dc07a19ce09c in term 1.
I20260807 22:45:20.750701 17540 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c, cb89d319b0cd4afaa08bbdd5915446ca; no voters:
I20260807 22:45:20.750914 17730 raft_consensus.cc:2781] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:20.751122 17730 raft_consensus.cc:686] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 LEADER]: Becoming Leader. State: Replica: becdaa6bebe442ebba24dc07a19ce09c, State: Running, Role: LEADER
I20260807 22:45:20.751345 17730 consensus_queue.cc:237] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:20.754127 17422 catalog_manager.cc:5495] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c reported cstate change: term changed from 0 to 1, leader changed from <none> to becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193). New cstate: current_term: 1 leader_uuid: "becdaa6bebe442ebba24dc07a19ce09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } health_report { overall_health: HEALTHY } } }
W20260807 22:45:20.754431 17720 tablet.cc:2369] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:20.770499 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.767672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.26.189.193:34377)
0807 22:45:20.767778 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:20.767782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:20.767794 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:45:20.767917 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:20.767919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:20.767969 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:20.768049 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:20.768220 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.768684 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.769394 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:20.769584 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:20.769646 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:20.769825 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:20.770203 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:20.770281 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:20.770350 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 22:45:20.770399 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:45:20.773830 17863 raft_consensus.cc:1261] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Refusing update from remote peer becdaa6bebe442ebba24dc07a19ce09c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:20.773829 18070 raft_consensus.cc:1261] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Refusing update from remote peer becdaa6bebe442ebba24dc07a19ce09c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:20.774473 18172 consensus_queue.cc:1035] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:20.774659 17730 consensus_queue.cc:1035] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:20.782626 18178 mvcc.cc:205] Tried to move back new op lower bound from 7316040584284377088 to 7316040584199770112. Current Snapshot: MvccSnapshot[applied={T|T < 7316040584284377088}]
I20260807 22:45:20.789189 17421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:20.789331 17421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:20.791507 17421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Sending DeleteTablet for 3 replicas of tablet 14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:20.792173 17613 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fc478ae61749df89efd68a3094fd7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:20 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:46044
I20260807 22:45:20.792191 17832 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fc478ae61749df89efd68a3094fd7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:20 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:39176
I20260807 22:45:20.792199 18043 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fc478ae61749df89efd68a3094fd7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:20 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:36542
I20260807 22:45:20.792343 17734 tablet_replica.cc:330] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: stopping tablet replica
I20260807 22:45:20.792447 17734 raft_consensus.cc:2229] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:45:20.792737 17734 raft_consensus.cc:2258] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:20.793121 18194 tablet_replica.cc:330] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: stopping tablet replica
I20260807 22:45:20.793316 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 17482
I20260807 22:45:20.793594 17734 ts_tablet_manager.cc:1905] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:20.792822 18193 tablet_replica.cc:330] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: stopping tablet replica
I20260807 22:45:20.799978 18193 raft_consensus.cc:2229] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:20.800305 18193 raft_consensus.cc:2258] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:20.800561 18194 raft_consensus.cc:2229] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:20.800840 18194 raft_consensus.cc:2258] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:20.802502 18193 ts_tablet_manager.cc:1905] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:20.803282 18194 ts_tablet_manager.cc:1905] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:20.805274 18193 ts_tablet_manager.cc:1918] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:20.805375 18193 log.cc:1196] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:20.805739 18193 ts_tablet_manager.cc:1939] T 14fc478ae61749df89efd68a3094fd7d P cb89d319b0cd4afaa08bbdd5915446ca: Deleting consensus metadata
I20260807 22:45:20.806109 18194 ts_tablet_manager.cc:1918] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:20.806205 18194 log.cc:1196] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:20.806552 18194 ts_tablet_manager.cc:1939] T 14fc478ae61749df89efd68a3094fd7d P eb710f7bdb5e4fe3b55748d3cb3e9e38: Deleting consensus metadata
I20260807 22:45:20.807103 17401 catalog_manager.cc:4841] TS cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): tablet 14fc478ae61749df89efd68a3094fd7d (table test-table [id=c784873fa20b41dda7dbc7664173c8a9]) successfully deleted
I20260807 22:45:20.807619 17396 catalog_manager.cc:4841] TS eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217): tablet 14fc478ae61749df89efd68a3094fd7d (table test-table [id=c784873fa20b41dda7dbc7664173c8a9]) successfully deleted
W20260807 22:45:20.808144 17402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:34377 (error 108)
I20260807 22:45:20.808525 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 17727
I20260807 22:45:20.810712 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:20.810423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34377 (local address 127.0.0.1:46068)
0807 22:45:20.810543 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:20.810627 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 22:45:20.810870 17402 catalog_manager.cc:4584] TS becdaa6bebe442ebba24dc07a19ce09c (127.26.189.193:34377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14fc478ae61749df89efd68a3094fd7d: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34377: connect: Connection refused (error 111)
I20260807 22:45:20.816555 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 17942
I20260807 22:45:20.829651 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 17366
I20260807 22:45:20.840863 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43995
--webserver_interface=127.26.189.254
--webserver_port=46071
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:20.951148 18197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:20.951397 18197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:20.951452 18197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:20.955400 18197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:20.955454 18197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:20.955480 18197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:20.955511 18197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:20.955536 18197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:20.960703 18197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=46071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:20.961686 18197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:20.962692 18197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:20.970715 18203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.970717 18202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:20.970765 18208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:20.970750 18197 server_base.cc:1034] running on GCE node
I20260807 22:45:20.971520 18197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:20.972487 18197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:20.973640 18197 hybrid_clock.cc:648] HybridClock initialized: now 1786142720973626 us; error 26 us; skew 500 ppm
Aug 07 22:45:20 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142720, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:20.976259 18197 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:20.977066 18197 webserver.cc:460] Webserver started at http://127.26.189.254:46071/ using document root <none> and password file <none>
I20260807 22:45:20.977566 18197 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:20.977625 18197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:20.981233 18197 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:20.983297 18220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:20.984460 18197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:20.984575 18197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "abfbe055bf6d493e8604254395f48e7e"
format_stamp: "Formatted at 2026-08-07 22:45:18 on dist-test-slave-4tdj"
server_key: "4f998f9caf8c3374a3bff57195e91012"
server_key_iv: "d05469979e34b87300dea8ab99e7c65b"
server_key_version: "encryptionkey@0"
I20260807 22:45:20.985082 18197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:21.009589 18197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:21.012369 18197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:21.012542 18197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:21.019142 18197 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43995
I20260807 22:45:21.019408 18291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43995 every 8 connection(s)
I20260807 22:45:21.020449 18197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:21.025806 18293 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:45:21.027091 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 18197
I20260807 22:45:21.027597 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:34377
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:43995
--webserver_port=44359
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:21.027976 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap starting.
I20260807 22:45:21.045449 18293 log.cc:824] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:21.056874 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:21.057216 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: Bootstrap complete.
I20260807 22:45:21.061090 18293 raft_consensus.cc:348] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:21.061467 18293 raft_consensus.cc:729] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: abfbe055bf6d493e8604254395f48e7e, State: Initialized, Role: FOLLOWER
I20260807 22:45:21.061937 18293 consensus_queue.cc:260] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:21.062072 18293 raft_consensus.cc:388] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:21.062146 18293 raft_consensus.cc:482] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:21.062243 18293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:45:21.064139 18293 raft_consensus.cc:504] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:21.064433 18293 leader_election.cc:302] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abfbe055bf6d493e8604254395f48e7e; no voters:
I20260807 22:45:21.064718 18293 leader_election.cc:288] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:45:21.064841 18297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:45:21.065047 18297 raft_consensus.cc:686] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [term 3 LEADER]: Becoming Leader. State: Replica: abfbe055bf6d493e8604254395f48e7e, State: Running, Role: LEADER
I20260807 22:45:21.065481 18297 consensus_queue.cc:237] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } }
I20260807 22:45:21.065923 18293 sys_catalog.cc:564] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:21.066416 18297 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader abfbe055bf6d493e8604254395f48e7e. Latest consensus state: current_term: 3 leader_uuid: "abfbe055bf6d493e8604254395f48e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } } }
I20260807 22:45:21.066526 18297 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:21.066910 18298 sys_catalog.cc:455] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "abfbe055bf6d493e8604254395f48e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abfbe055bf6d493e8604254395f48e7e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43995 } } }
I20260807 22:45:21.067046 18298 sys_catalog.cc:458] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:21.071290 18324 catalog_manager.cc:1259] Loaded cluster ID: 6f2a3a8e12f4465c9dded8db2f36f6a0
I20260807 22:45:21.071353 18324 catalog_manager.cc:1552] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: loading cluster ID for follower catalog manager: success
W20260807 22:45:21.072518 18324 catalog_manager.cc:1576] T 00000000000000000000000000000000 P abfbe055bf6d493e8604254395f48e7e: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260807 22:45:21.072587 18324 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260807 22:45:21.072645 18308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:21.075347 18308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:21.075832 18308 catalog_manager.cc:1259] Loaded cluster ID: 6f2a3a8e12f4465c9dded8db2f36f6a0
I20260807 22:45:21.075892 18308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 22:45:21.076491 18308 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd5c44ad9ad google::LogMessage::Fail() at ??:0
@ 0x7fd5c44b088a google::LogMessage::SendToLog() at ??:0
@ 0x7fd5c44ad4de google::LogMessage::Flush() at ??:0
@ 0x7fd5c44aea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd5cedecfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd5cededd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd5cedea579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd5cee2e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd5c66ff3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd5c66ffc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd5c67013f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd5c66f0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd5c7d346db start_thread at ??:0
@ 0x7fd5c2fab61f clone at ??:0
W20260807 22:45:21.169713 18295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:21.170068 18295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:21.170140 18295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:21.176102 18295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:21.176178 18295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:21.176265 18295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:21.184273 18295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:34377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=44359
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:21.185708 18295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:21.187050 18295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:21.197979 18357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:21.198076 18361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:21.198180 18295 server_base.cc:1034] running on GCE node
W20260807 22:45:21.197968 18358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:21.198706 18295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:21.199323 18295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:21.200493 18295 hybrid_clock.cc:648] HybridClock initialized: now 1786142721200467 us; error 42 us; skew 500 ppm
Aug 07 22:45:21 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142721, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:21.203270 18295 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:21.204159 18295 webserver.cc:460] Webserver started at http://127.26.189.193:44359/ using document root <none> and password file <none>
I20260807 22:45:21.204712 18295 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:21.204777 18295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:21.208719 18295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:45:21.210973 18372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.212287 18295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:45:21.212396 18295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "becdaa6bebe442ebba24dc07a19ce09c"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "a498820ef5e2d74eb023466e44ba49da"
server_key_iv: "d662729649f002223ea02a598c553c8d"
server_key_version: "encryptionkey@0"
I20260807 22:45:21.212944 18295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:21.247557 18295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:21.250699 18295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:21.250877 18295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:21.251468 18295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:21.252630 18385 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:45:21.258051 18295 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:45:21.258122 18295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 22:45:21.258193 18295 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:45:21.259887 18295 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:45:21.259933 18295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 22:45:21.260260 18385 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap starting.
I20260807 22:45:21.266427 18385 log.cc:824] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:21.270037 18295 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:34377
I20260807 22:45:21.270074 18542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:34377 every 8 connection(s)
I20260807 22:45:21.270992 18295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:21.275804 18385 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:21.275884 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 18295
I20260807 22:45:21.276258 18385 tablet_bootstrap.cc:492] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Bootstrap complete.
I20260807 22:45:21.276268 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:43217
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:43995
--webserver_port=38119
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:21.276624 18385 ts_tablet_manager.cc:1397] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20260807 22:45:21.279667 18385 raft_consensus.cc:348] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:21.280185 18385 raft_consensus.cc:729] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: becdaa6bebe442ebba24dc07a19ce09c, State: Initialized, Role: FOLLOWER
I20260807 22:45:21.280673 18385 consensus_queue.cc:260] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:21.281672 18385 ts_tablet_manager.cc:1428] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c: Time spent starting tablet: real 0.005s user 0.004s sys 0.003s
W20260807 22:45:21.314339 18545 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:45:21.272846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:56701)
0807 22:45:21.273318 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.273370 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:21.274205 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:21.313384 (+ 39179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":126,"threads_started":1}
W20260807 22:45:21.314859 18543 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:43995 (0 consecutive failures): Network error: Failed to ping master at 127.26.189.254:43995: Client connection negotiation failed: client connection to 127.26.189.254:43995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:45:21.315613 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.315327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:47771)
0807 22:45:21.315462 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.315540 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 22:45:21.316200 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.315995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:45313)
0807 22:45:21.316095 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.316144 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 22:45:21.316361 18543 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:45:21.393397 18547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:21.393646 18547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:21.393687 18547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:21.397773 18547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:21.397830 18547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:21.397886 18547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:21.402995 18547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:43217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=38119
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:21.403899 18547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:21.404852 18547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:21.413829 18562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:21.413846 18557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:21.413846 18560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:21.414160 18547 server_base.cc:1034] running on GCE node
I20260807 22:45:21.414564 18547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:21.415158 18547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:21.416311 18547 hybrid_clock.cc:648] HybridClock initialized: now 1786142721416294 us; error 26 us; skew 500 ppm
Aug 07 22:45:21 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142721, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:21.418967 18547 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:21.419849 18547 webserver.cc:460] Webserver started at http://127.26.189.194:38119/ using document root <none> and password file <none>
I20260807 22:45:21.420351 18547 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:21.420411 18547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:21.424018 18547 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:21.426000 18573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.427129 18547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:21.427227 18547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "e52de5c7df9bb5979fccb029fbbf397f"
server_key_iv: "8e25b0142e0c8b32a950d66081bcb3fe"
server_key_version: "encryptionkey@0"
I20260807 22:45:21.427748 18547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:21.445390 18547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:21.448545 18547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:21.448740 18547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:21.449318 18547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:21.450127 18547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:21.450174 18547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.450215 18547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:21.450244 18547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.458860 18547 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:43217
I20260807 22:45:21.458911 18752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:43217 every 8 connection(s)
I20260807 22:45:21.459808 18547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:21.463294 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.461743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:58519)
0807 22:45:21.462177 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.462616 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
W20260807 22:45:21.463711 18753 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:43995 (0 consecutive failures): Network error: Failed to ping master at 127.26.189.254:43995: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
I20260807 22:45:21.464242 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.464047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:40185)
0807 22:45:21.464140 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.464188 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:45:21.464871 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.464690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:53215)
0807 22:45:21.464781 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.464825 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 22:45:21.465014 18753 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:45:21.465416 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 18547
I20260807 22:45:21.465834 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:39785
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:43995
--webserver_port=38213
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:21.572662 18759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:21.572888 18759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:21.572927 18759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:21.576769 18759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:21.576817 18759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:21.576869 18759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:21.581862 18759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:39785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=38213
--tserver_master_addrs=127.26.189.254:43995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:21.582774 18759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:21.583693 18759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:21.592440 18770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:21.592514 18769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:21.592520 18772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:21.592724 18759 server_base.cc:1034] running on GCE node
I20260807 22:45:21.593158 18759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:21.593755 18759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:21.594918 18759 hybrid_clock.cc:648] HybridClock initialized: now 1786142721594901 us; error 28 us; skew 500 ppm
Aug 07 22:45:21 dist-test-slave-4tdj krb5kdc[16554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142721, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:21.597447 18759 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:21.598253 18759 webserver.cc:460] Webserver started at http://127.26.189.195:38213/ using document root <none> and password file <none>
I20260807 22:45:21.598778 18759 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:21.598838 18759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:21.602360 18759 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260807 22:45:21.604266 18782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.605443 18759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:21.605541 18759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "cb89d319b0cd4afaa08bbdd5915446ca"
format_stamp: "Formatted at 2026-08-07 22:45:19 on dist-test-slave-4tdj"
server_key: "669e960fe0b44d5f3280dfade316791e"
server_key_iv: "678650151b10310a73ef536ff7295716"
server_key_version: "encryptionkey@0"
I20260807 22:45:21.606057 18759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:21.622856 18759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:21.625700 18759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:21.625860 18759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:21.626423 18759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:21.627218 18759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:21.627264 18759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.627305 18759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:21.627334 18759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:21.635488 18759 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:39785
I20260807 22:45:21.635521 18968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:39785 every 8 connection(s)
I20260807 22:45:21.636413 18759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:21.639930 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.638413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:48779)
0807 22:45:21.638853 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.639261 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
W20260807 22:45:21.640355 18972 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:43995 (0 consecutive failures): Network error: Failed to ping master at 127.26.189.254:43995: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
I20260807 22:45:21.640882 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.640681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:41909)
0807 22:45:21.640782 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.640828 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 22:45:21.641453 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:21.641264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:44825)
0807 22:45:21.641362 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:21.641404 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 22:45:21.641623 18972 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:45:21.642011 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 18759
I20260807 22:45:22.317294 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.316879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:46531)
0807 22:45:22.317134 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:22.317197 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260807 22:45:22.465920 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.465519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:47147)
0807 22:45:22.465763 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:22.465819 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
I20260807 22:45:22.487805 18983 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:22.487983 18983 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:22.489156 18983 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
Aug 07 22:45:22 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142721, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.194@KRBTEST.COM
Aug 07 22:45:22 dist-test-slave-4tdj krb5kdc[16554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142721, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.195@KRBTEST.COM
I20260807 22:45:22.502214 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.489343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:39785 (local address 127.26.189.193:48213)
0807 22:45:22.489525 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:22.489539 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:22.489645 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:22.491324 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:22.491330 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:22.491844 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:22.492323 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:22.492337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.493588 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.493591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:22.494522 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:22.494531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.495193 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.495197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:22.495413 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:22.496028 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:22.496040 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:22.497813 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:22.499925 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:22.499931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:22.499944 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:22.500255 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:22.500564 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:22.500569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:22.500706 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:22.501272 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:22.501275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:22.501491 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 22:45:22.501651 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:45:22.501981 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":119,"spinlock_wait_cycles":23040}
I20260807 22:45:22.502226 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.489507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:52891 (local address 127.26.189.194:43217)
0807 22:45:22.490013 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:22.490032 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:22.490062 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:45:22.490722 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:22.490727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:22.491005 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:22.491267 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:22.492482 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.493410 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.494634 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.495067 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.495220 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:22.497997 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:22.498001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:22.498011 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:22.498037 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:22.499932 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500367 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:22.500372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:22.500376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:22.500480 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500798 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:22.500802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:22.500804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:22.501155 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:22.501263 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:22.501795 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 22:45:22.502048 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":86,"threads_started":1}
I20260807 22:45:22.502266 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.489584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:48213 (local address 127.26.189.195:39785)
0807 22:45:22.489876 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:22.489889 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:22.489923 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:45:22.490626 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:22.490632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:22.490973 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:22.491243 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:22.492481 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.493448 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.494628 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.495070 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.495225 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:22.497985 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:22.497990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:22.498001 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:22.498027 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:22.499815 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500361 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:22.500367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:22.500371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:22.500474 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500794 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:22.500797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:22.500799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:22.501168 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:22.501285 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:22.501829 (+ 544us) server_negotiation.cc:299] Negotiation successful
0807 22:45:22.502078 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260807 22:45:22.502255 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.489364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:43217 (local address 127.26.189.193:52891)
0807 22:45:22.489671 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:22.489685 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:22.489785 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:22.491325 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:22.491330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:22.491853 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:22.492324 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:22.492337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.493551 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.493555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:22.494524 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:22.494534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:22.495179 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:22.495182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:22.495414 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:22.495964 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:22.495979 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:22.497509 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:22.500032 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:22.500035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:22.500266 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:22.500568 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:22.500571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:22.500573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:22.500709 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:22.501265 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:22.501271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:22.501491 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 22:45:22.501650 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 22:45:22.502067 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":323,"spinlock_wait_cycles":27008,"thread_start_us":68,"threads_started":1}
W20260807 22:45:22.505447 18376 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
W20260807 22:45:22.506047 18373 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:22.506122 18373 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c; no voters: cb89d319b0cd4afaa08bbdd5915446ca, eb710f7bdb5e4fe3b55748d3cb3e9e38
I20260807 22:45:22.506371 18983 raft_consensus.cc:2726] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:45:22.642473 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:22.642064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:45533)
0807 22:45:22.642333 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:22.642383 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260807 22:45:23.318318 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:23.317899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:48169)
0807 22:45:23.318142 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:23.318194 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260807 22:45:23.467000 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:23.466650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:57963)
0807 22:45:23.466864 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:23.466914 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260807 22:45:23.643477 18993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:23.643073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:51843)
0807 22:45:23.643323 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:23.643377 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260807 22:45:24.299902 18995 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:24.300038 18995 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:24.300477 18995 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
W20260807 22:45:24.301266 18373 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
W20260807 22:45:24.301370 18376 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:24.301496 18376 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c; no voters: cb89d319b0cd4afaa08bbdd5915446ca, eb710f7bdb5e4fe3b55748d3cb3e9e38
I20260807 22:45:24.301664 18995 raft_consensus.cc:2726] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:45:24.319259 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:24.318887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:54717)
0807 22:45:24.319128 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:24.319179 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260807 22:45:24.468009 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:24.467629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:36259)
0807 22:45:24.467858 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:24.467910 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260807 22:45:24.644469 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:24.644083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:60761)
0807 22:45:24.644335 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:24.644384 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
I20260807 22:45:25.320256 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:25.319849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:55389)
0807 22:45:25.320107 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:25.320162 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260807 22:45:25.469056 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:25.468684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:40715)
0807 22:45:25.468918 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:25.468968 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260807 22:45:25.645526 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:25.645118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:45885)
0807 22:45:25.645381 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:25.645440 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":100,"threads_started":1}
I20260807 22:45:26.129007 19003 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:26.129145 19003 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:26.129528 19003 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
W20260807 22:45:26.130282 18373 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
W20260807 22:45:26.130438 18376 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:26.130527 18376 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c; no voters: cb89d319b0cd4afaa08bbdd5915446ca, eb710f7bdb5e4fe3b55748d3cb3e9e38
I20260807 22:45:26.130666 19003 raft_consensus.cc:2726] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:45:26.321504 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:26.321036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:44525)
0807 22:45:26.321354 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:26.321410 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":106,"threads_started":1}
I20260807 22:45:26.470221 19005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:26.469798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:59565)
0807 22:45:26.470058 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:26.470129 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260807 22:45:26.646618 19006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:26.646205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:50307)
0807 22:45:26.646482 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:26.646533 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":102,"threads_started":1}
I20260807 22:45:27.322608 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:27.322155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:50449)
0807 22:45:27.322439 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:27.322497 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":91,"threads_started":1}
I20260807 22:45:27.471282 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:27.470860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:53133)
0807 22:45:27.471114 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:27.471171 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260807 22:45:27.647645 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:27.647270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:50513)
0807 22:45:27.647496 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:27.647552 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260807 22:45:28.323637 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:28.323231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:55243)
0807 22:45:28.323472 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:28.323523 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260807 22:45:28.472422 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:28.471995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:49863)
0807 22:45:28.472273 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:28.472326 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":76,"threads_started":1}
I20260807 22:45:28.648756 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:28.648278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:40181)
0807 22:45:28.648592 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:28.648655 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":73,"threads_started":1}
I20260807 22:45:29.324775 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:29.324347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:56783)
0807 22:45:29.324617 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:29.324679 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260807 22:45:29.473487 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:29.473074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:42803)
0807 22:45:29.473333 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:29.473389 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260807 22:45:29.649761 19020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:29.649387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:49809)
0807 22:45:29.649618 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:29.649671 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260807 22:45:30.325871 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:30.325470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:41037)
0807 22:45:30.325721 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:30.325776 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260807 22:45:30.474599 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:30.474161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:48605)
0807 22:45:30.474448 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:30.474504 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":79,"threads_started":1}
I20260807 22:45:30.650822 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:30.650440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:39289)
0807 22:45:30.650674 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:30.650728 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260807 22:45:30.993124 19025 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:30.993273 19025 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:30.993681 19025 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
W20260807 22:45:30.994400 18376 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
W20260807 22:45:30.994499 18373 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:30.994596 18373 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c; no voters: cb89d319b0cd4afaa08bbdd5915446ca, eb710f7bdb5e4fe3b55748d3cb3e9e38
I20260807 22:45:30.994741 19025 raft_consensus.cc:2726] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:45:31.327231 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:31.326788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:58593)
0807 22:45:31.327068 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:31.327135 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
I20260807 22:45:31.475844 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:31.475403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:53153)
0807 22:45:31.475673 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:31.475733 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":73,"threads_started":1}
I20260807 22:45:31.652067 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:31.651584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:42373)
0807 22:45:31.651866 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:31.651937 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260807 22:45:32.328328 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:32.327943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:42023)
0807 22:45:32.328179 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:32.328233 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":76,"threads_started":1}
I20260807 22:45:32.477039 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:32.476521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:45917)
0807 22:45:32.476837 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:32.476905 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":105,"threads_started":1}
I20260807 22:45:32.653237 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:32.652808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:52431)
0807 22:45:32.653080 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:32.653139 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260807 22:45:33.329402 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:33.329003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:46727)
0807 22:45:33.329249 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:33.329306 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260807 22:45:33.419195 19036 raft_consensus.cc:482] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:33.419332 19036 raft_consensus.cc:504] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb89d319b0cd4afaa08bbdd5915446ca" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 39785 } } peers { permanent_uuid: "eb710f7bdb5e4fe3b55748d3cb3e9e38" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 43217 } } peers { permanent_uuid: "becdaa6bebe442ebba24dc07a19ce09c" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34377 } }
I20260807 22:45:33.419924 19036 leader_election.cc:288] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785), eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217)
W20260807 22:45:33.420440 18376 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cb89d319b0cd4afaa08bbdd5915446ca (127.26.189.195:39785): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
W20260807 22:45:33.420554 18373 leader_election.cc:341] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb710f7bdb5e4fe3b55748d3cb3e9e38 (127.26.189.194:43217): Not found: Tablet not found: 14fc478ae61749df89efd68a3094fd7d
I20260807 22:45:33.420639 18373 leader_election.cc:302] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: becdaa6bebe442ebba24dc07a19ce09c; no voters: cb89d319b0cd4afaa08bbdd5915446ca, eb710f7bdb5e4fe3b55748d3cb3e9e38
I20260807 22:45:33.420787 19036 raft_consensus.cc:2726] T 14fc478ae61749df89efd68a3094fd7d P becdaa6bebe442ebba24dc07a19ce09c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:45:33.478134 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:33.477731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:35517)
0807 22:45:33.477983 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:33.478044 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260807 22:45:33.654428 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:33.653938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:51991)
0807 22:45:33.654220 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:33.654311 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260807 22:45:34.330525 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:34.330090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:50093)
0807 22:45:34.330366 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:34.330427 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260807 22:45:34.479202 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:34.478794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:49459)
0807 22:45:34.479019 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:34.479073 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260807 22:45:34.655498 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:34.655130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:43771)
0807 22:45:34.655355 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:34.655407 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260807 22:45:35.331700 19043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.331249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:46327)
0807 22:45:35.331523 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.331600 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
I20260807 22:45:35.480300 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.479881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:48277)
0807 22:45:35.480147 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.480202 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
I20260807 22:45:35.656596 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:35.656190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:50371)
0807 22:45:35.656444 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:35.656501 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260807 22:45:36.332999 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:36.332559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.193:58535)
0807 22:45:36.332816 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:36.332883 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":70,"threads_started":1}
I20260807 22:45:36.481504 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:36.481117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.194:43629)
0807 22:45:36.481361 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:36.481416 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260807 22:45:36.643797 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 18295
I20260807 22:45:36.658010 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 18547
I20260807 22:45:36.658073 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:36.657565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43995 (local address 127.26.189.195:52537)
0807 22:45:36.657918 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:36.657974 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:43995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":82,"threads_started":1}
I20260807 22:45:36.669565 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 18759
2026-08-07T22:45:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19351 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:45:36 dist-test-slave-4tdj krb5kdc[19057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:45:36 dist-test-slave-4tdj krb5kdc[19057](info): set up 2 sockets
Aug 07 22:45:36 dist-test-slave-4tdj krb5kdc[19057](info): commencing operation
krb5kdc: starting...
W20260807 22:45:37.810988 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:45:37 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:45:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:45:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:45:37.932992 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38543
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:38.042603 19081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:38.042860 19081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:38.042914 19081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:38.046900 19081 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:45:38.046950 19081 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:45:38.046973 19081 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:45:38.046989 19081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:38.047014 19081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:38.047046 19081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:38.047077 19081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:38.047102 19081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:38.052248 19081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:38.053256 19081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:38.054217 19081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:38.062372 19090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.062379 19087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.062384 19088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:38.062450 19081 server_base.cc:1034] running on GCE node
I20260807 22:45:38.063202 19081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:38.064231 19081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:38.065387 19081 hybrid_clock.cc:648] HybridClock initialized: now 1786142738065365 us; error 35 us; skew 500 ppm
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:38.072150 19081 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:38.073148 19081 webserver.cc:460] Webserver started at http://127.26.189.254:38729/ using document root <none> and password file <none>
I20260807 22:45:38.073693 19081 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:38.073760 19081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:38.073978 19081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:38.076056 19081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "6795713f08b94bdabc1b5d3f4c4adf99"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "f0b536266c538d8fbfb017f1163620fb"
server_key_iv: "ca23d7ccdbd2eba641ba19646245dc6d"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.076618 19081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "6795713f08b94bdabc1b5d3f4c4adf99"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "f0b536266c538d8fbfb017f1163620fb"
server_key_iv: "ca23d7ccdbd2eba641ba19646245dc6d"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.080353 19081 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:45:38.082824 19099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.083972 19081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:38.084100 19081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "6795713f08b94bdabc1b5d3f4c4adf99"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "f0b536266c538d8fbfb017f1163620fb"
server_key_iv: "ca23d7ccdbd2eba641ba19646245dc6d"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.084205 19081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:38.103895 19081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:38.106781 19081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:38.106994 19081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:38.114539 19081 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38543
I20260807 22:45:38.114818 19177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38543 every 8 connection(s)
I20260807 22:45:38.115581 19081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:38.118427 19182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:38.118757 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 19081
I20260807 22:45:38.118903 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:45:38.119256 27383 external_mini_cluster.cc:1428] Setting key da9f1c0c4679a7a5959a3ddb3c1c0ad1
I20260807 22:45:38.125520 19182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap starting.
I20260807 22:45:38.128191 19182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Neither blocks nor log segments found. Creating new log.
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:38.128955 19182 log.cc:824] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:38.130956 19182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: No bootstrap required, opened a new log
I20260807 22:45:38.133687 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.120703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.26.189.254:38543)
0807 22:45:38.121124 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.121144 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.121181 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:45:38.121900 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.121919 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.122248 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.122590 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:38.123490 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.124492 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.125683 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.126197 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.126385 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.129255 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:38.129263 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:38.129282 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:38.129314 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:38.130955 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.131422 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.131426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.131431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.131599 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.131871 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.131873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.131874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.132247 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:38.132386 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.132683 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.132940 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":125,"threads_started":1}
I20260807 22:45:38.134891 19182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:38.135125 19182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:38.135195 19182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6795713f08b94bdabc1b5d3f4c4adf99, State: Initialized, Role: FOLLOWER
I20260807 22:45:38.135623 19182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:38.135763 19182 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:38.135843 19182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:38.135944 19182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:38.136850 19182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:38.137181 19182 leader_election.cc:302] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6795713f08b94bdabc1b5d3f4c4adf99; no voters:
I20260807 22:45:38.137504 19182 leader_election.cc:288] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:45:38.137629 19188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:38.137847 19188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 LEADER]: Becoming Leader. State: Replica: 6795713f08b94bdabc1b5d3f4c4adf99, State: Running, Role: LEADER
I20260807 22:45:38.138190 19188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:38.138710 19182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:38.139832 19192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6795713f08b94bdabc1b5d3f4c4adf99. Latest consensus state: current_term: 1 leader_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } } }
I20260807 22:45:38.139956 19192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:38.140255 19191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } } }
I20260807 22:45:38.140350 19201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:38.140419 19191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:38.143491 19201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:38.148893 19201 catalog_manager.cc:1347] Generated new cluster ID: 20a93f8b603d4f32a90c57d10fe17125
I20260807 22:45:38.148976 19201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:38.154347 19201 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:45:38.155252 19201 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:38.163535 19201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Generated new TSK 0
I20260807 22:45:38.164201 19201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:45:38.194401 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38543
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:38.305365 19220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:38.305611 19220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:38.305655 19220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:38.309653 19220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:38.309710 19220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:38.309767 19220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:38.314896 19220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:38.315897 19220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:38.316885 19220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:38.325937 19227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.325941 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.326004 19226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:38.326205 19220 server_base.cc:1034] running on GCE node
I20260807 22:45:38.326656 19220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:38.327248 19220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:38.328416 19220 hybrid_clock.cc:648] HybridClock initialized: now 1786142738328395 us; error 35 us; skew 500 ppm
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:38.331082 19220 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:38.331905 19220 webserver.cc:460] Webserver started at http://127.26.189.193:37065/ using document root <none> and password file <none>
I20260807 22:45:38.332420 19220 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:38.332485 19220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:38.332701 19220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:38.334689 19220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "df99f144f71040828f099bc42ff437e3"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "028c6873d9d1913092a16c19a2ea41db"
server_key_iv: "0bd4f5691bfe0a2ac32c5ef07347290d"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.335211 19220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "df99f144f71040828f099bc42ff437e3"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "028c6873d9d1913092a16c19a2ea41db"
server_key_iv: "0bd4f5691bfe0a2ac32c5ef07347290d"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.338711 19220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:45:38.341243 19238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.342478 19220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:38.342581 19220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "df99f144f71040828f099bc42ff437e3"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "028c6873d9d1913092a16c19a2ea41db"
server_key_iv: "0bd4f5691bfe0a2ac32c5ef07347290d"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.342694 19220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:38.368424 19220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:38.371567 19220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:38.371763 19220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:38.372327 19220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:38.373134 19220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:38.373183 19220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.373234 19220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:38.373260 19220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.382082 19220 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42647
I20260807 22:45:38.382107 19399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42647 every 8 connection(s)
I20260807 22:45:38.382997 19220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:38.391173 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 19220
I20260807 22:45:38.391314 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:45:38.391656 27383 external_mini_cluster.cc:1428] Setting key 28a64259f3fbbb1ab88b463388c06bf1
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:38.396353 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.384767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37889 (local address 127.26.189.254:38543)
0807 22:45:38.384896 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.384899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.385721 (+ 822us) server_negotiation.cc:407] Connection header received
0807 22:45:38.387093 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.387096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.387177 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.387249 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:38.388461 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.388999 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.390120 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.390398 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.390494 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.393056 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:38.393059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:38.393061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:38.393083 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:38.394436 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.394909 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.394913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.394914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.395001 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.395309 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.395313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.395314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.395492 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:38.395576 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.396065 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.396193 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:45:38.397122 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.385109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:37889)
0807 22:45:38.385570 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.385613 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.386879 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.387423 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.387429 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.387787 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:38.388312 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.388328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.389122 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.389125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.389995 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.390005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.390546 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.390550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.390707 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.391236 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:38.391257 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:38.392904 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:38.394552 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:38.394558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:38.394571 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:38.394814 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:38.395082 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:38.395086 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:38.395088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:38.395222 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:38.395590 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:38.395596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:38.395825 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.395998 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.396310 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260807 22:45:38.398221 19400 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38543
I20260807 22:45:38.398494 19400 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:38.398962 19400 heartbeater.cc:511] Master 127.26.189.254:38543 requested a full tablet report, sending...
I20260807 22:45:38.400346 19124 ts_manager.cc:194] Registered new tserver with Master: df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647)
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:45:38.401495 19124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:37889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:45:38.417243 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38543
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:38.526242 19407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:38.526520 19407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:38.526564 19407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:38.530483 19407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:38.530534 19407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:38.530588 19407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:38.535689 19407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:38.536635 19407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:38.537559 19407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:38.546797 19415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.546798 19418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.546799 19416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:38.547223 19407 server_base.cc:1034] running on GCE node
I20260807 22:45:38.547580 19407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:38.548166 19407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:38.549346 19407 hybrid_clock.cc:648] HybridClock initialized: now 1786142738549328 us; error 30 us; skew 500 ppm
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:38.552047 19407 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:38.552958 19407 webserver.cc:460] Webserver started at http://127.26.189.194:36881/ using document root <none> and password file <none>
I20260807 22:45:38.553476 19407 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:38.553541 19407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:38.553735 19407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:38.555765 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "9d708dc6518c4faabd162b92c4a59d12"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "b73217191d60033a81acc859e152e618"
server_key_iv: "356daf0298b0c91b3d416bddac79fbe4"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.556291 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "9d708dc6518c4faabd162b92c4a59d12"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "b73217191d60033a81acc859e152e618"
server_key_iv: "356daf0298b0c91b3d416bddac79fbe4"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.559927 19407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:45:38.562163 19430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.563374 19407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:38.563494 19407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "9d708dc6518c4faabd162b92c4a59d12"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "b73217191d60033a81acc859e152e618"
server_key_iv: "356daf0298b0c91b3d416bddac79fbe4"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.563586 19407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:38.574283 19407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:38.576934 19407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:38.577104 19407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:38.577631 19407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:38.578455 19407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:38.578512 19407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.578594 19407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:38.578624 19407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.587069 19407 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:33071
I20260807 22:45:38.587095 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:33071 every 8 connection(s)
I20260807 22:45:38.587893 19407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:38.593494 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 19407
I20260807 22:45:38.593621 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:45:38.593951 27383 external_mini_cluster.cc:1428] Setting key 9d183d33374a2910ab86e273cb78cc32
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:38.601320 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.589539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:37421 (local address 127.26.189.254:38543)
0807 22:45:38.589682 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.589686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.590590 (+ 904us) server_negotiation.cc:407] Connection header received
0807 22:45:38.591828 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.591831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.591880 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.591949 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:38.593119 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.593696 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.595080 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.595393 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.595486 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.597926 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:38.597929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:38.597931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:38.597955 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:38.599315 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.599894 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.599897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.599898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.599966 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.600312 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.600315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.600316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.600486 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:38.600559 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.601105 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.601203 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 22:45:38.602414 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.589991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:37421)
0807 22:45:38.590445 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.590485 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.591617 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.592092 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.592099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.592477 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:38.592976 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.592990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.593837 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.593841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.594962 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.594972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.595545 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.595550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.595703 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.596231 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:38.596247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:38.597770 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:38.599427 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:38.599433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:38.599446 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:38.599805 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:38.600042 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:38.600046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:38.600048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:38.600227 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:38.600579 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:38.600585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:38.600870 (+ 285us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.601082 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.601394 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":120,"threads_started":1}
I20260807 22:45:38.603502 19591 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38543
I20260807 22:45:38.603714 19591 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:45:38.604158 19591 heartbeater.cc:511] Master 127.26.189.254:38543 requested a full tablet report, sending...
I20260807 22:45:38.605108 19124 ts_manager.cc:194] Registered new tserver with Master: 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
I20260807 22:45:38.605679 19124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:37421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:45:38.621099 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38543
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:38.729224 19597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:38.729516 19597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:38.729593 19597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:38.733564 19597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:38.733618 19597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:38.733672 19597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:38.738850 19597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:38.739853 19597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:38.740856 19597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:38.750206 19605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:38.750324 19597 server_base.cc:1034] running on GCE node
W20260807 22:45:38.750326 19604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:38.750245 19607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:38.750979 19597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:38.751593 19597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:38.752759 19597 hybrid_clock.cc:648] HybridClock initialized: now 1786142738752737 us; error 35 us; skew 500 ppm
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:38.755461 19597 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:38.756281 19597 webserver.cc:460] Webserver started at http://127.26.189.195:39135/ using document root <none> and password file <none>
I20260807 22:45:38.756824 19597 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:38.756886 19597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:38.757087 19597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:45:38.759109 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "b4a3866ecf4247b2901b6da5d14fb594"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "864cc34180606d20be08320f64d01e9d"
server_key_iv: "b465d3644ced1ec555a33a4d4d92bcbf"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.759617 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "b4a3866ecf4247b2901b6da5d14fb594"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "864cc34180606d20be08320f64d01e9d"
server_key_iv: "b465d3644ced1ec555a33a4d4d92bcbf"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.763191 19597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:45:38.765440 19622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.766625 19597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:38.766727 19597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "b4a3866ecf4247b2901b6da5d14fb594"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "864cc34180606d20be08320f64d01e9d"
server_key_iv: "b465d3644ced1ec555a33a4d4d92bcbf"
server_key_version: "encryptionkey@0"
I20260807 22:45:38.766821 19597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:38.793772 19597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:38.796451 19597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:38.796620 19597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:38.797235 19597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:38.798039 19597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:38.798084 19597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.798125 19597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:38.798152 19597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:38.806387 19597 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:38231
I20260807 22:45:38.806435 19785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:38231 every 8 connection(s)
I20260807 22:45:38.807260 19597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:45:38 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142738, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:38.817296 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 19597
I20260807 22:45:38.817413 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:45:38.817663 27383 external_mini_cluster.cc:1428] Setting key ac66e96baa4a470a942218254efa34b7
I20260807 22:45:38.819516 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.808918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:55431 (local address 127.26.189.254:38543)
0807 22:45:38.809073 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.809078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.809832 (+ 754us) server_negotiation.cc:407] Connection header received
0807 22:45:38.811103 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.811106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.811152 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.811228 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:38.812323 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.812855 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.813887 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.814086 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.814152 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.816329 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:38.816332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:38.816334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:38.816359 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:38.817631 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.818079 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.818083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.818085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.818172 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.818516 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.818519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.818521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.818697 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:38.818779 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.819245 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.819362 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:45:38.820238 19788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.809272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:55431)
0807 22:45:38.809695 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.809735 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.810921 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.811368 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.811373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.811733 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:38.812183 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.812197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.812949 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.812952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.813800 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.813809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.814187 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.814189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.814366 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.814835 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:38.814849 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:38.816180 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:38.817732 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:38.817738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:38.817751 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:38.817986 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:38.818253 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:38.818256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:38.818259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:38.818430 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:38.818792 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:38.818798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:38.819016 (+ 218us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.819163 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.819458 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":114,"threads_started":1}
I20260807 22:45:38.821174 19786 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38543
I20260807 22:45:38.821384 19786 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:38.821841 19786 heartbeater.cc:511] Master 127.26.189.254:38543 requested a full tablet report, sending...
I20260807 22:45:38.822804 19124 ts_manager.cc:194] Registered new tserver with Master: b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231)
I20260807 22:45:38.823343 19124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:55431
I20260807 22:45:38.831116 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:38.839555 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.832935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.26.189.254:38543)
0807 22:45:38.833064 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.833068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.833218 (+ 150us) server_negotiation.cc:407] Connection header received
0807 22:45:38.833326 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.833329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.833380 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.833441 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:38.834094 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.834637 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.835419 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.835629 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.835714 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.836959 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:38.836963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:38.836965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:38.836986 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:38.837965 (+ 979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.838417 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.838421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.838422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.838498 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:38.838735 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:38.838738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:38.838739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:38.838941 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:38.839064 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.839301 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.839408 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:45:38.842103 19124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:45:38.844964 19124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:45:38.858825 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.853130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46206 (local address 127.26.189.195:38231)
0807 22:45:38.853452 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.853463 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.853842 (+ 379us) server_negotiation.cc:407] Connection header received
0807 22:45:38.853921 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.853926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.854137 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.854286 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:38.854748 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.855827 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.857599 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.858163 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.858273 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.858430 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.858587 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.858685 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260807 22:45:38.858819 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.852829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:38231 (local address 127.0.0.1:46206)
0807 22:45:38.853526 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.853576 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.853713 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.854440 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.854446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.854467 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:38.854611 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.854619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.855982 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.855985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.857468 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.857478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.858272 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.858276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.858411 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.858449 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.858549 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.858670 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":87,"spinlock_wait_cycles":4608,"thread_start_us":63,"threads_started":1}
I20260807 22:45:38.859036 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.852954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42647 (local address 127.0.0.1:38652)
0807 22:45:38.853526 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.853576 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.853713 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.854630 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.854633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.854653 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:38.854753 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.854760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.855916 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.855919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.857467 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.857478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.858636 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.858640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.858745 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.858766 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.858825 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.858892 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":112,"spinlock_wait_cycles":6656,"thread_start_us":93,"threads_started":1}
I20260807 22:45:38.859078 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.853437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.26.189.193:42647)
0807 22:45:38.853700 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.853716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.853739 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:45:38.853850 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.853856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.854479 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.854887 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:38.855014 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.855805 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.857609 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.858449 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.858702 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.858823 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.858870 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.858950 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
I20260807 22:45:38.859807 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.853828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.0.0.1:40584)
0807 22:45:38.854219 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.854231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.854326 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.855293 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.855296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.855314 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:38.855414 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.855419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.856491 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.856494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.857615 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.857625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.858636 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.858640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.859548 (+ 908us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.859572 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.859627 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.859687 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260807 22:45:38.860657 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.854060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.26.189.194:33071)
0807 22:45:38.854873 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.854891 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.854911 (+ 20us) server_negotiation.cc:407] Connection header received
0807 22:45:38.854981 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.854986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.855152 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.855290 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:38.855524 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.856367 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.857730 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.858518 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.860072 (+ 1554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.860207 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.860257 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.860517 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":115,"threads_started":1}
I20260807 22:45:38.861866 19694 tablet_service.cc:1467] Processing CreateTablet for tablet ad276fa251574c6281388d6f65868c79 (DEFAULT_TABLE table=test-table [id=d24ea5ee00dc4010bb053f9cea9282d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:38.861865 19310 tablet_service.cc:1467] Processing CreateTablet for tablet ad276fa251574c6281388d6f65868c79 (DEFAULT_TABLE table=test-table [id=d24ea5ee00dc4010bb053f9cea9282d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:38.862993 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad276fa251574c6281388d6f65868c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:38.863010 19694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad276fa251574c6281388d6f65868c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:38.862807 19504 tablet_service.cc:1467] Processing CreateTablet for tablet ad276fa251574c6281388d6f65868c79 (DEFAULT_TABLE table=test-table [id=d24ea5ee00dc4010bb053f9cea9282d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:38.863642 19504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad276fa251574c6281388d6f65868c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:38.869421 19813 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Bootstrap starting.
I20260807 22:45:38.870745 19817 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Bootstrap starting.
I20260807 22:45:38.872085 19813 tablet_bootstrap.cc:654] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:38.872396 19817 tablet_bootstrap.cc:654] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:38.872812 19813 log.cc:824] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:38.873387 19817 log.cc:824] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:38.874019 19814 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Bootstrap starting.
I20260807 22:45:38.875005 19813 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: No bootstrap required, opened a new log
I20260807 22:45:38.875190 19813 ts_tablet_manager.cc:1397] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 22:45:38.875275 19817 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: No bootstrap required, opened a new log
I20260807 22:45:38.875453 19817 ts_tablet_manager.cc:1397] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 22:45:38.876518 19814 tablet_bootstrap.cc:654] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:38.877300 19814 log.cc:824] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:38.878181 19813 raft_consensus.cc:348] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.878435 19813 raft_consensus.cc:374] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:38.878490 19813 raft_consensus.cc:729] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d708dc6518c4faabd162b92c4a59d12, State: Initialized, Role: FOLLOWER
I20260807 22:45:38.878886 19813 consensus_queue.cc:260] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.879016 19817 raft_consensus.cc:348] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.879276 19817 raft_consensus.cc:374] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:38.879299 19814 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: No bootstrap required, opened a new log
I20260807 22:45:38.879336 19817 raft_consensus.cc:729] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df99f144f71040828f099bc42ff437e3, State: Initialized, Role: FOLLOWER
I20260807 22:45:38.879460 19814 ts_tablet_manager.cc:1397] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:45:38.879769 19813 ts_tablet_manager.cc:1428] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260807 22:45:38.879753 19817 consensus_queue.cc:260] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.880113 19591 heartbeater.cc:503] Master 127.26.189.254:38543 was elected leader, sending a full tablet report...
I20260807 22:45:38.880544 19400 heartbeater.cc:503] Master 127.26.189.254:38543 was elected leader, sending a full tablet report...
I20260807 22:45:38.880738 19817 ts_tablet_manager.cc:1428] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260807 22:45:38.883116 19814 raft_consensus.cc:348] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.883378 19814 raft_consensus.cc:374] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:38.883427 19814 raft_consensus.cc:729] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a3866ecf4247b2901b6da5d14fb594, State: Initialized, Role: FOLLOWER
I20260807 22:45:38.883762 19814 consensus_queue.cc:260] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
W20260807 22:45:38.884061 19402 tablet.cc:2369] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:38.884604 19814 ts_tablet_manager.cc:1428] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:45:38.884919 19786 heartbeater.cc:503] Master 127.26.189.254:38543 was elected leader, sending a full tablet report...
I20260807 22:45:38.915956 19826 raft_consensus.cc:482] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:38.916163 19826 raft_consensus.cc:504] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.917212 19826 leader_election.cc:288] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
I20260807 22:45:38.921043 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.917563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:43111 (local address 127.26.189.194:33071)
0807 22:45:38.917700 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.917704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.917984 (+ 280us) server_negotiation.cc:407] Connection header received
0807 22:45:38.918110 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.918113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.918158 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.918229 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:38.918504 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.918977 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.920115 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.920504 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.920630 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.920686 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.920856 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.920925 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:45:38.921075 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.917428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.26.189.193:43111)
0807 22:45:38.917892 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.917906 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.918013 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.918251 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.918254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.918269 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:38.918399 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.918404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.919083 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.919086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.919979 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.919988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.920633 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.920637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.920739 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.920762 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.920829 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.920897 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":230,"thread_start_us":100,"threads_started":1}
I20260807 22:45:38.921082 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.917555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:37849 (local address 127.26.189.195:38231)
0807 22:45:38.917694 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.917700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.917910 (+ 210us) server_negotiation.cc:407] Connection header received
0807 22:45:38.918040 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.918043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.918087 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.918150 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:38.918438 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.918955 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.920128 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.920548 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.920658 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.920708 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.920893 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.920962 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:45:38.921101 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.917428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:38231 (local address 127.26.189.193:37849)
0807 22:45:38.917824 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:38.917839 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:38.917933 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:38.918178 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:38.918180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:38.918196 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:38.918327 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.918334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.919072 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.919075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.919990 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:38.920001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.920665 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.920669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:38.920787 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.920804 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:45:38.920869 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:45:38.920937 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":89,"thread_start_us":104,"threads_started":1}
I20260807 22:45:38.921846 19530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad276fa251574c6281388d6f65868c79" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d708dc6518c4faabd162b92c4a59d12" is_pre_election: true
I20260807 22:45:38.921928 19722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad276fa251574c6281388d6f65868c79" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a3866ecf4247b2901b6da5d14fb594" is_pre_election: true
I20260807 22:45:38.922135 19530 raft_consensus.cc:2454] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df99f144f71040828f099bc42ff437e3 in term 0.
I20260807 22:45:38.922187 19722 raft_consensus.cc:2454] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df99f144f71040828f099bc42ff437e3 in term 0.
I20260807 22:45:38.922629 19242 leader_election.cc:302] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d708dc6518c4faabd162b92c4a59d12, df99f144f71040828f099bc42ff437e3; no voters:
I20260807 22:45:38.922894 19826 raft_consensus.cc:2781] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:38.922993 19826 raft_consensus.cc:482] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:38.923044 19826 raft_consensus.cc:3037] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:38.924034 19826 raft_consensus.cc:504] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.924345 19826 leader_election.cc:288] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 election: Requested vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
I20260807 22:45:38.924679 19722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad276fa251574c6281388d6f65868c79" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a3866ecf4247b2901b6da5d14fb594"
I20260807 22:45:38.924671 19530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad276fa251574c6281388d6f65868c79" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d708dc6518c4faabd162b92c4a59d12"
I20260807 22:45:38.924795 19530 raft_consensus.cc:3037] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:38.924795 19722 raft_consensus.cc:3037] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:38.926091 19530 raft_consensus.cc:2454] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df99f144f71040828f099bc42ff437e3 in term 1.
I20260807 22:45:38.926091 19722 raft_consensus.cc:2454] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df99f144f71040828f099bc42ff437e3 in term 1.
I20260807 22:45:38.926421 19241 leader_election.cc:302] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4a3866ecf4247b2901b6da5d14fb594, df99f144f71040828f099bc42ff437e3; no voters:
I20260807 22:45:38.926604 19826 raft_consensus.cc:2781] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:38.926815 19826 raft_consensus.cc:686] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 LEADER]: Becoming Leader. State: Replica: df99f144f71040828f099bc42ff437e3, State: Running, Role: LEADER
I20260807 22:45:38.927111 19826 consensus_queue.cc:237] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:38.930404 19124 catalog_manager.cc:5495] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to df99f144f71040828f099bc42ff437e3 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "df99f144f71040828f099bc42ff437e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } health_report { overall_health: UNKNOWN } } }
I20260807 22:45:38.955474 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:38.952039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38658 (local address 127.26.189.193:42647)
0807 22:45:38.952184 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:38.952189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:38.952203 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:45:38.952355 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:38.952357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:38.952412 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:38.952501 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:38.952748 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.953249 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.954087 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:38.954340 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:38.954428 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:38.954722 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:38.955116 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:38.955219 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:38.955286 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 22:45:38.955342 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:45:38.961262 19722 raft_consensus.cc:1261] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Refusing update from remote peer df99f144f71040828f099bc42ff437e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:38.961263 19530 raft_consensus.cc:1261] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Refusing update from remote peer df99f144f71040828f099bc42ff437e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:38.961901 19830 consensus_queue.cc:1035] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:38.962088 19826 consensus_queue.cc:1035] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:38.970252 19836 mvcc.cc:205] Tried to move back new op lower bound from 7316040658777968640 to 7316040658648068096. Current Snapshot: MvccSnapshot[applied={T|T < 7316040658777968640}]
I20260807 22:45:38.977114 19124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:38.977298 19124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:38.979729 19124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Sending DeleteTablet for 3 replicas of tablet ad276fa251574c6281388d6f65868c79
I20260807 22:45:38.980443 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 19220
I20260807 22:45:38.980432 19694 tablet_service.cc:1514] Processing DeleteTablet for tablet ad276fa251574c6281388d6f65868c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:46206
I20260807 22:45:38.980451 19310 tablet_service.cc:1514] Processing DeleteTablet for tablet ad276fa251574c6281388d6f65868c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:38652
I20260807 22:45:38.980506 19504 tablet_service.cc:1514] Processing DeleteTablet for tablet ad276fa251574c6281388d6f65868c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:40584
I20260807 22:45:38.986624 19849 tablet_replica.cc:330] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: stopping tablet replica
I20260807 22:45:38.986923 19849 raft_consensus.cc:2229] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:38.987208 19849 raft_consensus.cc:2258] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:38.988313 19853 tablet_replica.cc:330] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: stopping tablet replica
I20260807 22:45:38.988610 19853 raft_consensus.cc:2229] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:38.988878 19853 raft_consensus.cc:2258] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:38.989461 19849 ts_tablet_manager.cc:1905] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:38.990110 19853 ts_tablet_manager.cc:1905] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:38.992238 19849 ts_tablet_manager.cc:1918] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:38.992273 19853 ts_tablet_manager.cc:1918] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:38.992343 19849 log.cc:1196] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/ad276fa251574c6281388d6f65868c79
I20260807 22:45:38.992355 19853 log.cc:1196] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/ad276fa251574c6281388d6f65868c79
I20260807 22:45:38.992646 19853 ts_tablet_manager.cc:1939] T ad276fa251574c6281388d6f65868c79 P 9d708dc6518c4faabd162b92c4a59d12: Deleting consensus metadata
I20260807 22:45:38.992679 19849 ts_tablet_manager.cc:1939] T ad276fa251574c6281388d6f65868c79 P b4a3866ecf4247b2901b6da5d14fb594: Deleting consensus metadata
I20260807 22:45:38.993793 19106 catalog_manager.cc:4841] TS 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071): tablet ad276fa251574c6281388d6f65868c79 (table test-table [id=d24ea5ee00dc4010bb053f9cea9282d6]) successfully deleted
I20260807 22:45:38.993816 19103 catalog_manager.cc:4841] TS b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): tablet ad276fa251574c6281388d6f65868c79 (table test-table [id=d24ea5ee00dc4010bb053f9cea9282d6]) successfully deleted
W20260807 22:45:38.994902 19102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:42647 (error 108)
I20260807 22:45:38.995132 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 19407
I20260807 22:45:39.000885 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:39.000623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42647 (local address 127.0.0.1:38672)
0807 22:45:39.000748 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:39.000813 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 22:45:39.001047 19102 catalog_manager.cc:4584] TS df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad276fa251574c6281388d6f65868c79: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42647: connect: Connection refused (error 111)
I20260807 22:45:39.004696 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 19597
I20260807 22:45:39.022303 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 19081
I20260807 22:45:59.031405 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38543
--webserver_interface=127.26.189.254
--webserver_port=38729
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:45:59.142581 19858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:59.142824 19858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:59.142881 19858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:59.146795 19858 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:45:59.146842 19858 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:45:59.146864 19858 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:45:59.146880 19858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:45:59.146904 19858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:59.146938 19858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:59.146968 19858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:45:59.147003 19858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:45:59.152381 19858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:59.153393 19858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:59.154425 19858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:59.167397 19865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:59.167423 19868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:59.167409 19866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:59.167466 19858 server_base.cc:1034] running on GCE node
I20260807 22:45:59.168329 19858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:59.169426 19858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:59.170586 19858 hybrid_clock.cc:648] HybridClock initialized: now 1786142759170566 us; error 29 us; skew 500 ppm
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:59.176183 19858 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:45:59.177026 19858 webserver.cc:460] Webserver started at http://127.26.189.254:38729/ using document root <none> and password file <none>
I20260807 22:45:59.177582 19858 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:59.177660 19858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:59.181391 19858 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:59.183465 19877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.184609 19858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:59.184718 19858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "6795713f08b94bdabc1b5d3f4c4adf99"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "f0b536266c538d8fbfb017f1163620fb"
server_key_iv: "ca23d7ccdbd2eba641ba19646245dc6d"
server_key_version: "encryptionkey@0"
I20260807 22:45:59.185309 19858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:59.199414 19858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:59.201922 19858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:59.202095 19858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:59.208832 19858 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38543
I20260807 22:45:59.209172 19955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38543 every 8 connection(s)
I20260807 22:45:59.209877 19858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:45:59.214073 19956 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:45:59.215705 19956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap starting.
I20260807 22:45:59.219159 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 19858
I20260807 22:45:59.219585 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:42647
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:38543
--webserver_port=37065
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:59.232622 19956 log.cc:824] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:59.241473 19956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:59.241806 19956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap complete.
I20260807 22:45:59.245643 19956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:59.246016 19956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6795713f08b94bdabc1b5d3f4c4adf99, State: Initialized, Role: FOLLOWER
I20260807 22:45:59.246479 19956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:59.246610 19956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:45:59.246686 19956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:45:59.246786 19956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:45:59.248577 19956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:59.248885 19956 leader_election.cc:302] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6795713f08b94bdabc1b5d3f4c4adf99; no voters:
I20260807 22:45:59.249158 19956 leader_election.cc:288] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:45:59.249257 19960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:45:59.249480 19960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 LEADER]: Becoming Leader. State: Replica: 6795713f08b94bdabc1b5d3f4c4adf99, State: Running, Role: LEADER
I20260807 22:45:59.249760 19960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:45:59.250366 19956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:45:59.251032 19961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } } }
I20260807 22:45:59.251171 19961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:59.251709 19962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6795713f08b94bdabc1b5d3f4c4adf99. Latest consensus state: current_term: 2 leader_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } } }
I20260807 22:45:59.251854 19962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:45:59.252111 19971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:45:59.256916 19971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:45:59.257510 19971 catalog_manager.cc:1259] Loaded cluster ID: 20a93f8b603d4f32a90c57d10fe17125
I20260807 22:45:59.257573 19971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:45:59.259977 19971 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:45:59.260665 19971 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Loaded TSK: 0
I20260807 22:45:59.276783 19971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Generated new TSK 1
I20260807 22:45:59.277024 19971 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:45:59.337216 19958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:59.337443 19958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:59.337484 19958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:59.341809 19958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:59.341866 19958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:59.341928 19958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:45:59.347121 19958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:42647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=37065
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:59.348068 19958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:59.349004 19958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:59.358937 19995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:59.358950 19992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:59.359112 19958 server_base.cc:1034] running on GCE node
W20260807 22:45:59.358937 19993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:59.359616 19958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:59.360181 19958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:59.361388 19958 hybrid_clock.cc:648] HybridClock initialized: now 1786142759361358 us; error 47 us; skew 500 ppm
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:59.364074 19958 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:45:59.365011 19958 webserver.cc:460] Webserver started at http://127.26.189.193:37065/ using document root <none> and password file <none>
I20260807 22:45:59.365556 19958 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:59.365638 19958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:59.369324 19958 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:59.371443 20003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.372722 19958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:59.372828 19958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "df99f144f71040828f099bc42ff437e3"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "028c6873d9d1913092a16c19a2ea41db"
server_key_iv: "0bd4f5691bfe0a2ac32c5ef07347290d"
server_key_version: "encryptionkey@0"
I20260807 22:45:59.373334 19958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:59.388506 19958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:59.391217 19958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:59.391386 19958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:59.391933 19958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:59.392957 20013 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:45:59.396883 19958 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:45:59.396958 19958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260807 22:45:59.397025 19958 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:45:59.398705 19958 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:45:59.398751 19958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 22:45:59.398991 20013 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Bootstrap starting.
I20260807 22:45:59.408556 19958 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42647
I20260807 22:45:59.408619 20167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42647 every 8 connection(s)
I20260807 22:45:59.409449 19958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:45:59.416651 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 19958
I20260807 22:45:59.417158 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:33071
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:38543
--webserver_port=36881
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:45:59.417261 20013 log.cc:824] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:59.424604 20013 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:45:59.424959 20013 tablet_bootstrap.cc:492] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Bootstrap complete.
I20260807 22:45:59.426198 20013 ts_tablet_manager.cc:1397] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.004s
I20260807 22:45:59.427790 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.411597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:40829 (local address 127.26.189.254:38543)
0807 22:45:59.411999 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.412018 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.412329 (+ 311us) server_negotiation.cc:407] Connection header received
0807 22:45:59.413465 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.413481 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.413923 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.414316 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:59.415351 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.416806 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.418406 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.419066 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.419263 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.422172 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:59.422179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:59.422195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:59.422230 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:59.424239 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.424811 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.424818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.424825 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.424991 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.425429 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.425432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.425434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.425850 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:59.425977 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.426690 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.426982 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":122,"threads_started":1}
I20260807 22:45:59.427836 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.411474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:40829)
0807 22:45:59.412149 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.412201 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.413012 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.414378 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.414384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.414740 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:59.415179 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.415193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.417006 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.417009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.418261 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.418274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.419224 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.419228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.419460 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.420141 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:59.420161 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:59.421948 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:59.424366 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.424372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.424384 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.424704 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.425112 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.425116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.425118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.425317 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.425973 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:59.425980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:59.426281 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.426554 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.426886 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":172,"threads_started":1}
I20260807 22:45:59.428970 20013 raft_consensus.cc:348] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:59.429352 20013 raft_consensus.cc:729] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df99f144f71040828f099bc42ff437e3, State: Initialized, Role: FOLLOWER
I20260807 22:45:59.429812 20013 consensus_queue.cc:260] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } }
I20260807 22:45:59.430817 20013 ts_tablet_manager.cc:1428] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260807 22:45:59.430979 20168 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38543
I20260807 22:45:59.431201 20168 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:59.431735 20168 heartbeater.cc:511] Master 127.26.189.254:38543 requested a full tablet report, sending...
I20260807 22:45:59.434111 19897 ts_manager.cc:194] Registered new tserver with Master: df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647)
I20260807 22:45:59.437284 19897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:40829
I20260807 22:45:59.439894 20168 heartbeater.cc:503] Master 127.26.189.254:38543 was elected leader, sending a full tablet report...
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.193@KRBTEST.COM
I20260807 22:45:59.446189 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.436793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42647 (local address 127.0.0.1:36708)
0807 22:45:59.437483 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.437523 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.437686 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.438173 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.438179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.438537 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:59.438650 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.438656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.439560 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.439564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.440155 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.440163 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.440862 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.440866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.440977 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.441487 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:59.441503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:59.442971 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:59.444614 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.444624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.444629 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.444843 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.445113 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.445116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.445118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.445195 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.445620 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:59.445626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:59.445732 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.445843 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.446024 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":110,"threads_started":1}
I20260807 22:45:59.446234 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.436881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.26.189.193:42647)
0807 22:45:59.437236 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.437247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.437625 (+ 378us) server_negotiation.cc:407] Connection header received
0807 22:45:59.437831 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.437836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.438028 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.438155 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:59.438765 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.439453 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.440285 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.440724 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.440832 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.443105 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:59.443113 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:59.443122 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:59.443143 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:59.444504 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.444947 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.444952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.444953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.445027 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.445286 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.445290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.445291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.445503 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:59.445616 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.445938 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.446086 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":98,"threads_started":1}
I20260807 22:45:59.447404 20073 tablet_service.cc:1514] Processing DeleteTablet for tablet ad276fa251574c6281388d6f65868c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:36708
I20260807 22:45:59.448338 20183 tablet_replica.cc:330] stopping tablet replica
I20260807 22:45:59.448593 20183 raft_consensus.cc:2229] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:59.448812 20183 raft_consensus.cc:2258] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:59.449666 20183 ts_tablet_manager.cc:1905] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:59.453367 20183 ts_tablet_manager.cc:1918] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:59.453454 20183 log.cc:1196] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal/wals/ad276fa251574c6281388d6f65868c79
I20260807 22:45:59.453682 20183 ts_tablet_manager.cc:1939] T ad276fa251574c6281388d6f65868c79 P df99f144f71040828f099bc42ff437e3: Deleting consensus metadata
I20260807 22:45:59.455209 19879 catalog_manager.cc:4841] TS df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647): tablet ad276fa251574c6281388d6f65868c79 (table test-table [id=d24ea5ee00dc4010bb053f9cea9282d6]) successfully deleted
W20260807 22:45:59.532600 20175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:59.532878 20175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:59.532931 20175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:59.536985 20175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:59.537039 20175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:59.537106 20175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:45:59.542132 20175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:33071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=36881
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:59.543105 20175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:59.544085 20175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:59.553388 20194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:59.553439 20189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:59.553531 20175 server_base.cc:1034] running on GCE node
W20260807 22:45:59.553601 20192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:59.554036 20175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:59.554677 20175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:59.555833 20175 hybrid_clock.cc:648] HybridClock initialized: now 1786142759555813 us; error 31 us; skew 500 ppm
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:59.558393 20175 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:45:59.559226 20175 webserver.cc:460] Webserver started at http://127.26.189.194:36881/ using document root <none> and password file <none>
I20260807 22:45:59.559736 20175 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:59.559816 20175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:59.563449 20175 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:59.565486 20205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.566728 20175 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:45:59.566833 20175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "9d708dc6518c4faabd162b92c4a59d12"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "b73217191d60033a81acc859e152e618"
server_key_iv: "356daf0298b0c91b3d416bddac79fbe4"
server_key_version: "encryptionkey@0"
I20260807 22:45:59.567345 20175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:59.582144 20175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:59.584956 20175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:59.585140 20175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:59.585697 20175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:59.586563 20175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:59.586614 20175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.586668 20175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:59.586695 20175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.595408 20175 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:33071
I20260807 22:45:59.595436 20361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:33071 every 8 connection(s)
I20260807 22:45:59.596349 20175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:45:59.604480 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 20175
I20260807 22:45:59.604915 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:38231
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:38543
--webserver_port=39135
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:59.609639 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.598039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:51285 (local address 127.26.189.254:38543)
0807 22:45:59.598179 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.598183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.599026 (+ 843us) server_negotiation.cc:407] Connection header received
0807 22:45:59.600298 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.600301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.600366 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.600439 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:59.601622 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.602154 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.603287 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.603521 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.603613 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.606120 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:59.606124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:59.606127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:59.606157 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:59.607596 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.608170 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.608173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.608175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.608260 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.608550 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.608553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.608555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.608769 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:59.608860 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.609322 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.609471 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:45:59.610373 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.598442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:51285)
0807 22:45:59.598884 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.598937 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.600094 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.600661 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.600668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.601012 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:59.601478 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.601491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.602272 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.602275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.603168 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.603178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.603664 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.603668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.603834 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.604357 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:59.604373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:59.605944 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:59.607714 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.607721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.607735 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.608081 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.608338 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.608340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.608342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.608470 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.608864 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:59.608868 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:59.609103 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.609251 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.609494 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":118,"threads_started":1}
I20260807 22:45:59.611630 20362 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38543
I20260807 22:45:59.611850 20362 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:59.612308 20362 heartbeater.cc:511] Master 127.26.189.254:38543 requested a full tablet report, sending...
I20260807 22:45:59.613372 19897 ts_manager.cc:194] Registered new tserver with Master: 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
I20260807 22:45:59.614117 19897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:51285
W20260807 22:45:59.715418 20365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:45:59.715662 20365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:45:59.715700 20365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:45:59.719522 20365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:45:59.719565 20365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:45:59.719623 20365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:45:59.724584 20365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:38231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=39135
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:45:59.725514 20365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:45:59.726468 20365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:45:59.735630 20377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:59.735651 20372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:45:59.735625 20375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:45:59.735965 20365 server_base.cc:1034] running on GCE node
I20260807 22:45:59.736356 20365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:45:59.736949 20365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:45:59.738125 20365 hybrid_clock.cc:648] HybridClock initialized: now 1786142759738096 us; error 39 us; skew 500 ppm
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:45:59.740796 20365 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:45:59.741621 20365 webserver.cc:460] Webserver started at http://127.26.189.195:39135/ using document root <none> and password file <none>
I20260807 22:45:59.742168 20365 fs_manager.cc:359] Metadata directory not provided
I20260807 22:45:59.742250 20365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:45:59.746349 20365 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:45:59.748409 20389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.749599 20365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:45:59.749707 20365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "b4a3866ecf4247b2901b6da5d14fb594"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "864cc34180606d20be08320f64d01e9d"
server_key_iv: "b465d3644ced1ec555a33a4d4d92bcbf"
server_key_version: "encryptionkey@0"
I20260807 22:45:59.750269 20365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:45:59.763195 20365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:45:59.765806 20365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:45:59.765966 20365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:45:59.766575 20365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:45:59.767360 20365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:45:59.767408 20365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.767458 20365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:45:59.767485 20365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:45:59.776199 20365 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:38231
I20260807 22:45:59.776244 20552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:38231 every 8 connection(s)
I20260807 22:45:59.777220 20365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:45:59.782474 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 20365
Aug 07 22:45:59 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142759, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:45:59.791384 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.783098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.26.189.254:38543)
0807 22:45:59.783293 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.783298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.783546 (+ 248us) server_negotiation.cc:407] Connection header received
0807 22:45:59.783674 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.783676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.783719 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.783780 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:59.784701 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.785434 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.786561 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.786807 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.786909 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.788316 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:59.788320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:59.788323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:59.788359 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:59.789731 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.790145 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.790147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.790149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.790232 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.790516 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.790520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.790521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.790767 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:59.790956 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.791085 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.791209 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"spinlock_wait_cycles":4480,"thread_start_us":58,"threads_started":1}
I20260807 22:45:59.791692 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.779008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:43869 (local address 127.26.189.254:38543)
0807 22:45:59.779148 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.779152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.779948 (+ 796us) server_negotiation.cc:407] Connection header received
0807 22:45:59.781229 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.781233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.781289 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.781363 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:59.782558 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.783235 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.784857 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.785200 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.785315 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.787879 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:59.787884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:59.787887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:59.787917 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:59.789531 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.790068 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.790072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.790073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.790163 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.790523 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.790526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.790528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.790763 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:59.790848 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.791439 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.791545 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":54,"spinlock_wait_cycles":4480}
I20260807 22:45:59.792569 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.779373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:43869)
0807 22:45:59.779801 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.779843 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.781029 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.781554 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.781561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.781901 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:45:59.782418 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.782432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.783390 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.783394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.784746 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.784756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.785338 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.785341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.785482 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.786054 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:45:59.786073 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:45:59.787685 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:45:59.789643 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.789650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.789666 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.789974 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.790246 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:45:59.790250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:45:59.790252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:45:59.790433 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:45:59.790867 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:45:59.790872 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:45:59.791176 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.791369 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.791764 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260807 22:45:59.793584 20556 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38543
I20260807 22:45:59.793812 20556 heartbeater.cc:464] Registering TS with master...
I20260807 22:45:59.794281 20556 heartbeater.cc:511] Master 127.26.189.254:38543 requested a full tablet report, sending...
I20260807 22:45:59.795264 19900 ts_manager.cc:194] Registered new tserver with Master: b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231)
I20260807 22:45:59.795819 19900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:43869
I20260807 22:45:59.804814 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:45:59.808393 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.805382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.26.189.254:38543)
0807 22:45:59.805490 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.805494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.805682 (+ 188us) server_negotiation.cc:407] Connection header received
0807 22:45:59.805805 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.805808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.805899 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.805979 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:59.806158 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.806696 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.807507 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.807708 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.807774 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.807989 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:59.808116 (+ 127us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 22:45:59.808301 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60786: token expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 22:45:59.808446 20172 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60786: token expired
W20260807 22:45:59.808823 19789 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.189.254:38543)
I20260807 22:45:59.814936 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.809127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60796 (local address 127.26.189.254:38543)
0807 22:45:59.809223 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.809226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.809240 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:45:59.809324 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.809326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.809358 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.809414 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:45:59.809940 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.810441 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.811163 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.811350 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.811410 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.812535 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:45:59.812537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:45:59.812539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:45:59.812561 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:45:59.813656 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.814007 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.814009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.814010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.814070 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:45:59.814259 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:45:59.814260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:45:59.814262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:45:59.814440 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:45:59.814538 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.814754 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.814836 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:45:59.817106 19900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:45:59.820242 19900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:45:59.827463 20073 tablet_service.cc:1467] Processing CreateTablet for tablet 0bef3e38d0b144b3a4f539af3a4a9051 (DEFAULT_TABLE table=test-table [id=bb03b07203bd43b0bd53bba4dc758f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:59.828115 20073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bef3e38d0b144b3a4f539af3a4a9051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:59.830686 20013 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Bootstrap starting.
I20260807 22:45:59.831652 20013 tablet_bootstrap.cc:654] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:59.831892 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.826665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.0.0.1:46222)
0807 22:45:59.826975 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.826986 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.827061 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.827998 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.828001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.828025 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:59.828119 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.828125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.829482 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.829486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.830359 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.830371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.831419 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.831422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.831511 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.831614 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.831684 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.831750 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":71,"thread_start_us":91,"threads_started":1}
I20260807 22:45:59.831975 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.827056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.26.189.194:33071)
0807 22:45:59.827395 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.827407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.827425 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:45:59.827487 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.827491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.827636 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.827795 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:59.828304 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.829376 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.830511 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.831305 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.831425 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.831582 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.831722 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.831829 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":129,"threads_started":1}
I20260807 22:45:59.831899 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.826558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:38231 (local address 127.0.0.1:43918)
0807 22:45:59.826709 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.826725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.826802 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.827840 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.827843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.827862 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:59.827963 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.827969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.829417 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.829420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.830600 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.830611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.831533 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.831549 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.831681 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.831703 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.831752 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.831811 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":20}
I20260807 22:45:59.832777 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.826592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.26.189.195:38231)
0807 22:45:59.827288 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.827300 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.827324 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:45:59.827404 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.827410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.827637 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.827818 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:59.828084 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.829287 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.830778 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.831408 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.832365 (+ 957us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.832489 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.832548 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.832640 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":160,"threads_started":1}
I20260807 22:45:59.833679 20013 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: No bootstrap required, opened a new log
I20260807 22:45:59.833760 20013 ts_tablet_manager.cc:1397] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:45:59.834244 20013 raft_consensus.cc:348] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.834399 20013 raft_consensus.cc:374] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:59.834450 20013 raft_consensus.cc:729] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df99f144f71040828f099bc42ff437e3, State: Initialized, Role: FOLLOWER
I20260807 22:45:59.834600 20013 consensus_queue.cc:260] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.834822 20269 tablet_service.cc:1467] Processing CreateTablet for tablet 0bef3e38d0b144b3a4f539af3a4a9051 (DEFAULT_TABLE table=test-table [id=bb03b07203bd43b0bd53bba4dc758f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:59.835860 20269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bef3e38d0b144b3a4f539af3a4a9051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:59.835600 20463 tablet_service.cc:1467] Processing CreateTablet for tablet 0bef3e38d0b144b3a4f539af3a4a9051 (DEFAULT_TABLE table=test-table [id=bb03b07203bd43b0bd53bba4dc758f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:45:59.836378 20013 ts_tablet_manager.cc:1428] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:45:59.836661 20463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bef3e38d0b144b3a4f539af3a4a9051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:45:59.842495 20575 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Bootstrap starting.
I20260807 22:45:59.843250 20576 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Bootstrap starting.
I20260807 22:45:59.845029 20575 tablet_bootstrap.cc:654] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:59.845718 20576 tablet_bootstrap.cc:654] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Neither blocks nor log segments found. Creating new log.
I20260807 22:45:59.845723 20575 log.cc:824] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:59.846400 20576 log.cc:824] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Log is configured to *not* fsync() on all Append() calls
I20260807 22:45:59.847537 20575 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: No bootstrap required, opened a new log
I20260807 22:45:59.847710 20575 ts_tablet_manager.cc:1397] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:45:59.848115 20576 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: No bootstrap required, opened a new log
I20260807 22:45:59.848287 20576 ts_tablet_manager.cc:1397] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 22:45:59.850690 20575 raft_consensus.cc:348] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.850919 20575 raft_consensus.cc:374] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:59.850975 20575 raft_consensus.cc:729] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d708dc6518c4faabd162b92c4a59d12, State: Initialized, Role: FOLLOWER
I20260807 22:45:59.851240 20576 raft_consensus.cc:348] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.851481 20576 raft_consensus.cc:374] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:45:59.851426 20575 consensus_queue.cc:260] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.851544 20576 raft_consensus.cc:729] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a3866ecf4247b2901b6da5d14fb594, State: Initialized, Role: FOLLOWER
I20260807 22:45:59.851907 20576 consensus_queue.cc:260] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.852078 20362 heartbeater.cc:503] Master 127.26.189.254:38543 was elected leader, sending a full tablet report...
I20260807 22:45:59.852571 20556 heartbeater.cc:503] Master 127.26.189.254:38543 was elected leader, sending a full tablet report...
I20260807 22:45:59.852828 20576 ts_tablet_manager.cc:1428] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260807 22:45:59.852442 20575 ts_tablet_manager.cc:1428] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:45:59.887611 20583 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:59.887856 20583 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.888109 20178 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:45:59.888300 20178 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.889302 20583 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071), df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647)
I20260807 22:45:59.889367 20178 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
I20260807 22:45:59.894238 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.889511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.26.189.195:48715)
0807 22:45:59.889976 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.889990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.890085 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.890589 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.890591 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.890610 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:59.890699 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.890704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.891684 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.891687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.892899 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.892909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.893808 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.893811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.893925 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.893948 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.894013 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.894089 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":1}
I20260807 22:45:59.894539 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.890074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:48715 (local address 127.26.189.194:33071)
0807 22:45:59.890199 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.890203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.890215 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:45:59.890266 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.890269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.890333 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.890406 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:59.890821 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.891568 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.893039 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.893604 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.893705 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.893755 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.894355 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.894432 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:45:59.894757 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.889649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42647 (local address 127.26.189.195:55845)
0807 22:45:59.890126 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.890138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.890207 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.890533 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.890536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.890559 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:59.890694 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.890701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.891658 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.891661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.893428 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.893439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.894361 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.894458 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.894479 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.894545 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.894610 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":547,"thread_start_us":124,"threads_started":1}
I20260807 22:45:59.895190 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.890065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:55845 (local address 127.26.189.193:42647)
0807 22:45:59.890169 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.890173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.890187 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:45:59.890279 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.890281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.890340 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.890404 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:59.890823 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.891533 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.893574 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.894160 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894236 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.894648 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.894690 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.895078 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":527,"server-negotiator.queue_time_us":44}
I20260807 22:45:59.895334 20301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "b4a3866ecf4247b2901b6da5d14fb594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d708dc6518c4faabd162b92c4a59d12" is_pre_election: true
I20260807 22:45:59.895478 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.889511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:38231 (local address 127.26.189.193:36243)
0807 22:45:59.889761 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.889777 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.889881 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.891076 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.891078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.891094 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:59.891198 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.891205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.892116 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.892120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.893320 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.893330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.894224 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.895144 (+ 916us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.895181 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.895256 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.895335 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":72}
I20260807 22:45:59.895481 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.889649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.26.189.193:49627)
0807 22:45:59.890459 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.890485 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:45:59.890579 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:45:59.891158 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:45:59.891161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:45:59.891177 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:45:59.891320 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.891326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.892825 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.892829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.893926 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:45:59.893935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.894803 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:45:59.895159 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.895199 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 22:45:59.895256 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 22:45:59.895335 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":246,"thread_start_us":124,"threads_started":1}
I20260807 22:45:59.895532 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.890242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:49627 (local address 127.26.189.194:33071)
0807 22:45:59.890933 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.890947 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.890959 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:45:59.890998 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.891001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.891040 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.891857 (+ 817us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:59.891949 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.892698 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894077 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.894672 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894896 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.894942 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.895351 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.895422 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":129,"threads_started":1}
I20260807 22:45:59.895668 20301 raft_consensus.cc:2454] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a3866ecf4247b2901b6da5d14fb594 in term 0.
I20260807 22:45:59.895905 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d708dc6518c4faabd162b92c4a59d12" is_pre_election: true
I20260807 22:45:59.896023 20299 raft_consensus.cc:2454] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df99f144f71040828f099bc42ff437e3 in term 0.
I20260807 22:45:59.895988 20106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "b4a3866ecf4247b2901b6da5d14fb594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df99f144f71040828f099bc42ff437e3" is_pre_election: true
I20260807 22:45:59.896157 20394 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d708dc6518c4faabd162b92c4a59d12, b4a3866ecf4247b2901b6da5d14fb594; no voters:
I20260807 22:45:59.896201 20106 raft_consensus.cc:2454] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a3866ecf4247b2901b6da5d14fb594 in term 0.
I20260807 22:45:59.896413 20010 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d708dc6518c4faabd162b92c4a59d12, df99f144f71040828f099bc42ff437e3; no voters:
I20260807 22:45:59.896458 20583 raft_consensus.cc:2781] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:59.896544 20583 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:59.896648 20178 raft_consensus.cc:2781] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:45:59.896704 20178 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:45:59.896713 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.890715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:36243 (local address 127.26.189.195:38231)
0807 22:45:59.890844 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.890848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.890859 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:45:59.890890 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.890892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.890927 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.890989 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:45:59.891313 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.891991 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.893445 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.894037 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.894126 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.894166 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.896510 (+ 2344us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.896578 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":59}
I20260807 22:45:59.896731 20178 raft_consensus.cc:3037] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:59.897152 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a3866ecf4247b2901b6da5d14fb594" is_pre_election: true
I20260807 22:45:59.896590 20583 raft_consensus.cc:3037] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:59.897986 20178 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.898402 20178 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 election: Requested vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
I20260807 22:45:59.898604 20583 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.898706 20299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d708dc6518c4faabd162b92c4a59d12"
I20260807 22:45:59.898733 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "df99f144f71040828f099bc42ff437e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a3866ecf4247b2901b6da5d14fb594"
I20260807 22:45:59.898800 20299 raft_consensus.cc:3037] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:45:59.898856 20491 raft_consensus.cc:2379] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df99f144f71040828f099bc42ff437e3 in current term 1: Already voted for candidate b4a3866ecf4247b2901b6da5d14fb594 in this term.
I20260807 22:45:59.898962 20583 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [CANDIDATE]: Term 1 election: Requested vote from peers 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071), df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647)
I20260807 22:45:59.899490 20106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "b4a3866ecf4247b2901b6da5d14fb594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df99f144f71040828f099bc42ff437e3"
I20260807 22:45:59.899653 20106 raft_consensus.cc:2379] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b4a3866ecf4247b2901b6da5d14fb594 in current term 1: Already voted for candidate df99f144f71040828f099bc42ff437e3 in this term.
I20260807 22:45:59.899811 20301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef3e38d0b144b3a4f539af3a4a9051" candidate_uuid: "b4a3866ecf4247b2901b6da5d14fb594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d708dc6518c4faabd162b92c4a59d12"
I20260807 22:45:59.899889 20299 raft_consensus.cc:2454] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df99f144f71040828f099bc42ff437e3 in term 1.
I20260807 22:45:59.900161 20010 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d708dc6518c4faabd162b92c4a59d12, df99f144f71040828f099bc42ff437e3; no voters: b4a3866ecf4247b2901b6da5d14fb594
I20260807 22:45:59.900280 20394 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4a3866ecf4247b2901b6da5d14fb594; no voters: 9d708dc6518c4faabd162b92c4a59d12, df99f144f71040828f099bc42ff437e3
I20260807 22:45:59.900344 20178 raft_consensus.cc:2781] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:45:59.900480 20583 raft_consensus.cc:2726] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 22:45:59.900573 20178 raft_consensus.cc:686] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 LEADER]: Becoming Leader. State: Replica: df99f144f71040828f099bc42ff437e3, State: Running, Role: LEADER
I20260807 22:45:59.900817 20178 consensus_queue.cc:237] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:45:59.903797 19900 catalog_manager.cc:5495] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to df99f144f71040828f099bc42ff437e3 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "df99f144f71040828f099bc42ff437e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } health_report { overall_health: HEALTHY } } }
W20260807 22:45:59.910538 20170 tablet.cc:2369] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:45:59.914366 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.911359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.26.189.193:42647)
0807 22:45:59.911473 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:45:59.911476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:45:59.911534 (+ 58us) server_negotiation.cc:407] Connection header received
0807 22:45:59.911675 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:45:59.911677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:45:59.911725 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:45:59.911796 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:45:59.911959 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.912436 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.913192 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:45:59.913412 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:45:59.913501 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:45:59.913673 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:45:59.914046 (+ 373us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:45:59.914127 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:45:59.914179 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 22:45:59.914219 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:45:59.917801 20491 raft_consensus.cc:1261] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Refusing update from remote peer df99f144f71040828f099bc42ff437e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:59.917836 20301 raft_consensus.cc:1261] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Refusing update from remote peer df99f144f71040828f099bc42ff437e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:45:59.918336 20178 consensus_queue.cc:1035] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:59.918510 20590 consensus_queue.cc:1035] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:45:59.926148 20595 mvcc.cc:205] Tried to move back new op lower bound from 7316040744617852928 to 7316040744556130304. Current Snapshot: MvccSnapshot[applied={T|T < 7316040744617852928}]
I20260807 22:45:59.926342 20594 mvcc.cc:205] Tried to move back new op lower bound from 7316040744617852928 to 7316040744556130304. Current Snapshot: MvccSnapshot[applied={T|T < 7316040744617852928}]
I20260807 22:45:59.933024 19900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:59.933192 19900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:45:59.935462 19900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Sending DeleteTablet for 3 replicas of tablet 0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:45:59.936098 20463 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bef3e38d0b144b3a4f539af3a4a9051 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:59 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260807 22:45:59.936121 20269 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bef3e38d0b144b3a4f539af3a4a9051 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:59 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:46222
I20260807 22:45:59.936501 20073 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bef3e38d0b144b3a4f539af3a4a9051 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:45:59 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:36708
I20260807 22:45:59.936549 20606 tablet_replica.cc:330] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: stopping tablet replica
I20260807 22:45:59.936699 20183 tablet_replica.cc:330] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: stopping tablet replica
I20260807 22:45:59.936817 20183 raft_consensus.cc:2229] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:45:59.936836 20606 raft_consensus.cc:2229] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:59.936998 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 19958
I20260807 22:45:59.937070 20183 raft_consensus.cc:2258] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:59.956926 20607 tablet_replica.cc:330] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: stopping tablet replica
I20260807 22:45:59.957466 20607 raft_consensus.cc:2229] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:45:59.957939 20607 raft_consensus.cc:2258] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:59.958382 20606 raft_consensus.cc:2258] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:45:59.960412 20607 ts_tablet_manager.cc:1905] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:45:59.963025 20607 ts_tablet_manager.cc:1918] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:45:59.963122 20607 log.cc:1196] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:45:59.963485 20607 ts_tablet_manager.cc:1939] T 0bef3e38d0b144b3a4f539af3a4a9051 P b4a3866ecf4247b2901b6da5d14fb594: Deleting consensus metadata
I20260807 22:45:59.964294 20606 ts_tablet_manager.cc:1905] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 22:45:59.964823 19879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:42647 (error 108)
I20260807 22:45:59.965507 19880 catalog_manager.cc:4841] TS b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): tablet 0bef3e38d0b144b3a4f539af3a4a9051 (table test-table [id=bb03b07203bd43b0bd53bba4dc758f59]) successfully deleted
I20260807 22:45:59.965960 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 20175
I20260807 22:45:59.966082 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.965694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42647 (local address 127.0.0.1:36724)
0807 22:45:59.965901 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.965979 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260807 22:45:59.968932 19879 catalog_manager.cc:4584] TS df99f144f71040828f099bc42ff437e3 (127.26.189.193:42647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bef3e38d0b144b3a4f539af3a4a9051: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42647: connect: Connection refused (error 111)
I20260807 22:45:59.976326 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 20365
I20260807 22:45:59.976753 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:45:59.976433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.0.0.1:46238)
0807 22:45:59.976615 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:45:59.976668 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.194:33071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 22:45:59.986699 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 19858
I20260807 22:46:19.998737 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38543
--webserver_interface=127.26.189.254
--webserver_port=38729
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:20.112497 20609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:20.112771 20609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:20.112811 20609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:20.116793 20609 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:46:20.116847 20609 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:46:20.116861 20609 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:46:20.116875 20609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:20.116892 20609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:20.116914 20609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:20.116945 20609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:20.116971 20609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:20.122174 20609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:20.123202 20609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:20.124217 20609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:20.132822 20615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.132982 20617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.132890 20614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:20.132869 20609 server_base.cc:1034] running on GCE node
I20260807 22:46:20.133903 20609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:20.135052 20609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:20.136227 20609 hybrid_clock.cc:648] HybridClock initialized: now 1786142780136202 us; error 40 us; skew 500 ppm
Aug 07 22:46:20 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:20.146564 20609 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:46:20.147445 20609 webserver.cc:460] Webserver started at http://127.26.189.254:38729/ using document root <none> and password file <none>
I20260807 22:46:20.147974 20609 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:20.148036 20609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:20.152148 20609 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:46:20.154362 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.155660 20609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:20.155777 20609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "6795713f08b94bdabc1b5d3f4c4adf99"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "f0b536266c538d8fbfb017f1163620fb"
server_key_iv: "ca23d7ccdbd2eba641ba19646245dc6d"
server_key_version: "encryptionkey@0"
I20260807 22:46:20.156343 20609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:20.173215 20609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:20.176236 20609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:20.176412 20609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:20.183332 20609 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38543
I20260807 22:46:20.183581 20707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38543 every 8 connection(s)
I20260807 22:46:20.184379 20609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:20.187094 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 20609
I20260807 22:46:20.187613 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:42647
--local_ip_for_outbound_sockets=127.26.189.193
--tserver_master_addrs=127.26.189.254:38543
--webserver_port=37065
--webserver_interface=127.26.189.193
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:20.188974 20708 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:46:20.190732 20708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap starting.
I20260807 22:46:20.207867 20708 log.cc:824] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:20.218763 20708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:46:20.219105 20708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Bootstrap complete.
I20260807 22:46:20.222922 20708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:46:20.223312 20708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6795713f08b94bdabc1b5d3f4c4adf99, State: Initialized, Role: FOLLOWER
I20260807 22:46:20.223783 20708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:46:20.223903 20708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:20.223966 20708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:20.224058 20708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:46:20.225837 20708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:46:20.226146 20708 leader_election.cc:302] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6795713f08b94bdabc1b5d3f4c4adf99; no voters:
I20260807 22:46:20.226457 20708 leader_election.cc:288] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:46:20.226558 20713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:46:20.226783 20713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [term 3 LEADER]: Becoming Leader. State: Replica: 6795713f08b94bdabc1b5d3f4c4adf99, State: Running, Role: LEADER
I20260807 22:46:20.227124 20713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } }
I20260807 22:46:20.227640 20708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:20.228543 20715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6795713f08b94bdabc1b5d3f4c4adf99. Latest consensus state: current_term: 3 leader_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } } }
I20260807 22:46:20.228679 20715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:20.228991 20714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6795713f08b94bdabc1b5d3f4c4adf99" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38543 } } }
I20260807 22:46:20.229058 20726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:20.229080 20714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:20.233850 20726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:20.234531 20726 catalog_manager.cc:1259] Loaded cluster ID: 20a93f8b603d4f32a90c57d10fe17125
I20260807 22:46:20.234583 20726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:20.236265 20726 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:20.236840 20726 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Loaded TSK: 0
I20260807 22:46:20.236903 20726 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6795713f08b94bdabc1b5d3f4c4adf99: Loaded TSK: 1
F20260807 22:46:20.237469 20726 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6adb8049ad google::LogMessage::Fail() at ??:0
@ 0x7f6adb80788a google::LogMessage::SendToLog() at ??:0
@ 0x7f6adb8044de google::LogMessage::Flush() at ??:0
@ 0x7f6adb805a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6ade4c818d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6ade4c0f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6ae61697bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6ae614399c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6ae6144178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f6ae6185df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6ae61a482b std::function<>::operator()() at ??:0
@ 0x7f6ae6143b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6ae6144f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6ae6141579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6ae6185640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6adda563d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6adda56c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6adda583f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6adda47512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6adf08b6db start_thread at ??:0
@ 0x7f6ada30261f clone at ??:0
W20260807 22:46:20.318071 20710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:20.318358 20710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:20.318411 20710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:20.322475 20710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:20.322521 20710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:20.322580 20710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:20.327915 20710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:42647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=37065
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:20.328853 20710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:20.329911 20710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:20.339291 20782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.339296 20780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.339331 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:20.339535 20710 server_base.cc:1034] running on GCE node
I20260807 22:46:20.340152 20710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:20.340764 20710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:20.341961 20710 hybrid_clock.cc:648] HybridClock initialized: now 1786142780341933 us; error 44 us; skew 500 ppm
Aug 07 22:46:20 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:20.344844 20710 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:46:20.345948 20710 webserver.cc:460] Webserver started at http://127.26.189.193:37065/ using document root <none> and password file <none>
I20260807 22:46:20.346681 20710 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:20.346774 20710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:20.351629 20710 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 22:46:20.354210 20794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.355665 20710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:46:20.355786 20710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "df99f144f71040828f099bc42ff437e3"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "028c6873d9d1913092a16c19a2ea41db"
server_key_iv: "0bd4f5691bfe0a2ac32c5ef07347290d"
server_key_version: "encryptionkey@0"
I20260807 22:46:20.356464 20710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:20.385869 20710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:20.389070 20710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:20.389240 20710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:20.389835 20710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:20.390983 20805 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:46:20.396489 20710 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:46:20.396557 20710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 22:46:20.396638 20710 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:46:20.398258 20710 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:46:20.398325 20710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:20.398665 20805 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Bootstrap starting.
I20260807 22:46:20.408013 20710 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42647
I20260807 22:46:20.408231 20963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42647 every 8 connection(s)
I20260807 22:46:20.409021 20710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:20.415580 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 20710
I20260807 22:46:20.416013 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:33071
--local_ip_for_outbound_sockets=127.26.189.194
--tserver_master_addrs=127.26.189.254:38543
--webserver_port=36881
--webserver_interface=127.26.189.194
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:46:20.420908 20805 log.cc:824] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:20.430228 20805 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:46:20.430707 20805 tablet_bootstrap.cc:492] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Bootstrap complete.
I20260807 22:46:20.432278 20805 ts_tablet_manager.cc:1397] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Time spent bootstrapping tablet: real 0.034s user 0.017s sys 0.015s
I20260807 22:46:20.435638 20805 raft_consensus.cc:348] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:20.436153 20805 raft_consensus.cc:729] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df99f144f71040828f099bc42ff437e3, State: Initialized, Role: FOLLOWER
I20260807 22:46:20.436671 20805 consensus_queue.cc:260] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:20.437744 20805 ts_tablet_manager.cc:1428] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
W20260807 22:46:20.486621 20969 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:46:20.411377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:51361)
0807 22:46:20.411866 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.411919 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:20.412723 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:20.485799 (+ 73076us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":143,"threads_started":1}
W20260807 22:46:20.487300 20964 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:38543 (0 consecutive failures): Network error: Failed to ping master at 127.26.189.254:38543: Client connection negotiation failed: client connection to 127.26.189.254:38543: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:46:20.488049 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.487779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:49991)
0807 22:46:20.487908 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.487992 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 22:46:20.488714 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.488522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:53399)
0807 22:46:20.488622 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.488666 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 22:46:20.488922 20964 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:46:20.532676 20971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:20.532915 20971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:20.532956 20971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:20.536868 20971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:20.536916 20971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:20.536969 20971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:20.541887 20971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:33071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=36881
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:20.542817 20971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:20.543730 20971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:20.552724 20987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.552724 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.552822 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:20.552860 20971 server_base.cc:1034] running on GCE node
I20260807 22:46:20.553341 20971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:20.553948 20971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:20.555100 20971 hybrid_clock.cc:648] HybridClock initialized: now 1786142780555084 us; error 30 us; skew 500 ppm
Aug 07 22:46:20 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:20.557626 20971 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:46:20.558413 20971 webserver.cc:460] Webserver started at http://127.26.189.194:36881/ using document root <none> and password file <none>
I20260807 22:46:20.558899 20971 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:20.558955 20971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:20.562608 20971 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 22:46:20.564532 20996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.565740 20971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:20.565830 20971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "9d708dc6518c4faabd162b92c4a59d12"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "b73217191d60033a81acc859e152e618"
server_key_iv: "356daf0298b0c91b3d416bddac79fbe4"
server_key_version: "encryptionkey@0"
I20260807 22:46:20.566356 20971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:20.588918 20971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:20.591955 20971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:20.592113 20971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:20.592651 20971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:20.593681 21003 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260807 22:46:20.597393 21003 ts_tablet_manager.cc:1852] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 22:46:20.597806 21003 ts_tablet_manager.cc:1905] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:20.602025 21003 ts_tablet_manager.cc:1918] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:20.602123 21003 log.cc:1196] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:46:20.602542 21003 ts_tablet_manager.cc:1939] T 0bef3e38d0b144b3a4f539af3a4a9051 P 9d708dc6518c4faabd162b92c4a59d12: Deleting consensus metadata
I20260807 22:46:20.602845 20971 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 22:46:20.602905 20971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260807 22:46:20.602964 20971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:20.602995 20971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.611683 20971 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:33071
I20260807 22:46:20.611728 21152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:33071 every 8 connection(s)
I20260807 22:46:20.612538 20971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:20.615911 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.614644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:38981)
0807 22:46:20.615129 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.615238 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
W20260807 22:46:20.616276 21153 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:38543 (0 consecutive failures): Network error: Failed to ping master at 127.26.189.254:38543: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
I20260807 22:46:20.616809 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.616619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:38895)
0807 22:46:20.616710 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.616755 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 22:46:20.617358 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.617148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:54315)
0807 22:46:20.617261 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.617307 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:46:20.617496 21153 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:46:20.620864 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 20971
I20260807 22:46:20.621281 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:38231
--local_ip_for_outbound_sockets=127.26.189.195
--tserver_master_addrs=127.26.189.254:38543
--webserver_port=39135
--webserver_interface=127.26.189.195
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:20.729869 21156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:20.730109 21156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:20.730190 21156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:20.734210 21156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:20.734265 21156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:20.734354 21156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:20.739382 21156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:38231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=39135
--tserver_master_addrs=127.26.189.254:38543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:20.740316 21156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:20.741264 21156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:20.750345 21171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.750345 21166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:20.750365 21165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:20.750849 21156 server_base.cc:1034] running on GCE node
I20260807 22:46:20.751217 21156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:20.751801 21156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:20.752955 21156 hybrid_clock.cc:648] HybridClock initialized: now 1786142780752931 us; error 38 us; skew 500 ppm
Aug 07 22:46:20 dist-test-slave-4tdj krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:20.755566 21156 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:46:20.756357 21156 webserver.cc:460] Webserver started at http://127.26.189.195:39135/ using document root <none> and password file <none>
I20260807 22:46:20.756845 21156 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:20.756901 21156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:20.760663 21156 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:20.762698 21182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.763840 21156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:20.763947 21156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "b4a3866ecf4247b2901b6da5d14fb594"
format_stamp: "Formatted at 2026-08-07 22:45:38 on dist-test-slave-4tdj"
server_key: "864cc34180606d20be08320f64d01e9d"
server_key_iv: "b465d3644ced1ec555a33a4d4d92bcbf"
server_key_version: "encryptionkey@0"
I20260807 22:46:20.764443 21156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:20.779644 21156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:20.782840 21156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:20.783001 21156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:20.783562 21156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:20.784343 21156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:20.784387 21156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.784425 21156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:20.784451 21156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:20.792925 21156 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:38231
I20260807 22:46:20.792966 21346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:38231 every 8 connection(s)
I20260807 22:46:20.793862 21156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:20.797760 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.795932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:36099)
0807 22:46:20.796415 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.796537 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
I20260807 22:46:20.798007 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 21156
W20260807 22:46:20.798272 21347 heartbeater.cc:644] Failed to heartbeat to 127.26.189.254:38543 (0 consecutive failures): Network error: Failed to ping master at 127.26.189.254:38543: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
I20260807 22:46:20.798964 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.798704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:34387)
0807 22:46:20.798828 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.798890 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:46:20.799621 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:20.799398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:54705)
0807 22:46:20.799520 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:20.799569 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 22:46:20.799787 21347 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:46:21.490113 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:21.489641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:36979)
0807 22:46:21.489906 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:21.489963 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260807 22:46:21.618474 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:21.618020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:60939)
0807 22:46:21.618328 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:21.618389 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":93,"threads_started":1}
I20260807 22:46:21.800738 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:21.800317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:37555)
0807 22:46:21.800577 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:21.800634 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260807 22:46:22.096160 21360 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:22.096410 21360 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:22.097543 21360 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
Aug 07 22:46:22 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.194@KRBTEST.COM
Aug 07 22:46:22 dist-test-slave-4tdj krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.195@KRBTEST.COM
I20260807 22:46:22.114645 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.097777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:38231 (local address 127.26.189.193:33449)
0807 22:46:22.098096 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:22.098111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:22.098221 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:22.100296 (+ 2075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:22.100303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:22.100892 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:22.101687 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:22.101710 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.103714 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.103718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:22.105195 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:22.105218 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.106266 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.106270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:22.106650 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:22.107354 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:22.107376 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:22.109589 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:22.111822 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:22.111830 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:22.111845 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:22.112313 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:22.112724 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:22.112728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:22.112730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:22.112959 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:22.113770 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:22.113773 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:22.113928 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 22:46:22.114093 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 22:46:22.114449 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":234,"spinlock_wait_cycles":34432,"thread_start_us":72,"threads_started":1}
I20260807 22:46:22.114681 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.097978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:57815 (local address 127.26.189.194:33071)
0807 22:46:22.098406 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:22.098423 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:22.098471 (+ 48us) server_negotiation.cc:407] Connection header received
0807 22:46:22.099407 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:22.099415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:22.099831 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:22.100177 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:22.101893 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.103537 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.105354 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.106114 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.106325 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:22.109531 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:22.109537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:22.109550 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:22.109581 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:22.111690 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:22.112438 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:22.112445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:22.112458 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:22.112631 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:22.113058 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:22.113062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:22.113063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:22.113573 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:22.113702 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:22.114236 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 22:46:22.114525 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":108,"threads_started":1}
I20260807 22:46:22.114727 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.097967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:33449 (local address 127.26.189.195:38231)
0807 22:46:22.098406 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:22.098429 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:22.098476 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:46:22.099364 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:22.099373 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:22.099833 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:22.100177 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:22.101893 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.103537 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.105360 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.106114 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.106325 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:22.109801 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:22.109807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:22.109821 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:22.109859 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:22.111690 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:22.112438 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:22.112445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:22.112458 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:22.112631 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:22.113095 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:22.113097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:22.113099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:22.113585 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:22.113710 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:22.114237 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 22:46:22.114535 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":81,"threads_started":1}
I20260807 22:46:22.114741 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.097776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33071 (local address 127.26.189.193:57815)
0807 22:46:22.098205 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:22.098217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:22.098357 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:22.100296 (+ 1939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:22.100303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:22.100892 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:22.101687 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:22.101710 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.103714 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.103718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:22.105195 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:22.105220 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:22.106266 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:22.106270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:22.106685 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:22.107385 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:22.107397 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:22.109283 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:22.111823 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:22.111830 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:22.111845 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:22.112313 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:22.112724 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:22.112728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:22.112730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:22.112959 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:22.113697 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:22.113703 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:22.113933 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 22:46:22.114095 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 22:46:22.114543 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":479,"spinlock_wait_cycles":40960,"thread_start_us":61,"threads_started":1}
W20260807 22:46:22.117765 20802 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
W20260807 22:46:22.117913 20801 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:46:22.117985 20801 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df99f144f71040828f099bc42ff437e3; no voters: 9d708dc6518c4faabd162b92c4a59d12, b4a3866ecf4247b2901b6da5d14fb594
I20260807 22:46:22.118259 21360 raft_consensus.cc:2726] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:46:22.491168 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.490845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:46551)
0807 22:46:22.491013 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:22.491078 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 22:46:22.619544 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.619162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:45331)
0807 22:46:22.619384 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:22.619442 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":61,"threads_started":1}
I20260807 22:46:22.801800 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:22.801417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:37849)
0807 22:46:22.801658 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:22.801710 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
I20260807 22:46:23.492275 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:23.491882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:45139)
0807 22:46:23.492124 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:23.492177 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260807 22:46:23.620608 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:23.620192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:47491)
0807 22:46:23.620454 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:23.620517 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
I20260807 22:46:23.739666 21376 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:23.739801 21376 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:23.740190 21376 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
W20260807 22:46:23.741007 20802 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
W20260807 22:46:23.741130 20801 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:46:23.741214 20801 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df99f144f71040828f099bc42ff437e3; no voters: 9d708dc6518c4faabd162b92c4a59d12, b4a3866ecf4247b2901b6da5d14fb594
I20260807 22:46:23.741353 21376 raft_consensus.cc:2726] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:46:23.802909 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:23.802514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:59119)
0807 22:46:23.802745 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:23.802801 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":72,"threads_started":1}
I20260807 22:46:24.493469 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:24.493042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:49621)
0807 22:46:24.493290 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:24.493354 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260807 22:46:24.621694 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:24.621309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:44685)
0807 22:46:24.621546 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:24.621604 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260807 22:46:24.804011 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:24.803587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:46345)
0807 22:46:24.803852 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:24.803921 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260807 22:46:25.494910 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:25.494378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:37615)
0807 22:46:25.494717 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:25.494787 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260807 22:46:25.622861 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:25.622460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:54753)
0807 22:46:25.622717 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:25.622774 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260807 22:46:25.805277 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:25.804822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:34349)
0807 22:46:25.805088 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:25.805159 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
I20260807 22:46:26.496086 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:26.495620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:57297)
0807 22:46:26.495913 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:26.495984 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260807 22:46:26.623936 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:26.623568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:33779)
0807 22:46:26.623796 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:26.623849 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
I20260807 22:46:26.762503 21391 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:26.762640 21391 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:26.763062 21391 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
W20260807 22:46:26.763825 20801 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
W20260807 22:46:26.763962 20802 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:46:26.764031 20802 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df99f144f71040828f099bc42ff437e3; no voters: 9d708dc6518c4faabd162b92c4a59d12, b4a3866ecf4247b2901b6da5d14fb594
I20260807 22:46:26.764242 21391 raft_consensus.cc:2726] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:46:26.806433 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:26.806004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:60207)
0807 22:46:26.806256 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:26.806342 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260807 22:46:27.497198 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:27.496814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:45207)
0807 22:46:27.497045 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:27.497102 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260807 22:46:27.625058 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:27.624620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:34857)
0807 22:46:27.624904 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:27.624961 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":63,"threads_started":1}
I20260807 22:46:27.807677 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:27.807196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:43335)
0807 22:46:27.807510 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:27.807583 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":75,"threads_started":1}
I20260807 22:46:28.498368 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:28.497922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:54297)
0807 22:46:28.498177 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:28.498234 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260807 22:46:28.626155 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:28.625758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:33131)
0807 22:46:28.625998 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:28.626057 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260807 22:46:28.808840 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:28.808404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:38761)
0807 22:46:28.808685 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:28.808744 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260807 22:46:29.404105 21400 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:29.404237 21400 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:29.404610 21400 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
W20260807 22:46:29.405417 20802 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
W20260807 22:46:29.405567 20801 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:46:29.405634 20801 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df99f144f71040828f099bc42ff437e3; no voters: 9d708dc6518c4faabd162b92c4a59d12, b4a3866ecf4247b2901b6da5d14fb594
I20260807 22:46:29.405763 21400 raft_consensus.cc:2726] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:46:29.499500 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:29.499044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:53571)
0807 22:46:29.499344 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:29.499403 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":93,"threads_started":1}
I20260807 22:46:29.627267 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:29.626837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:59405)
0807 22:46:29.627106 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:29.627175 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260807 22:46:29.809937 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:29.809562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:41333)
0807 22:46:29.809791 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:29.809844 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260807 22:46:30.500802 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:30.500348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:40289)
0807 22:46:30.500647 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:30.500706 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":77,"threads_started":1}
I20260807 22:46:30.628373 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:30.627987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:50499)
0807 22:46:30.628230 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:30.628284 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260807 22:46:30.811163 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:30.810717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:60621)
0807 22:46:30.811016 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:30.811072 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":105,"threads_started":1}
I20260807 22:46:31.501901 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:31.501493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:38669)
0807 22:46:31.501751 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:31.501807 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260807 22:46:31.629480 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:31.629066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:55655)
0807 22:46:31.629330 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:31.629388 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260807 22:46:31.812264 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:31.811835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:49393)
0807 22:46:31.812106 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:31.812164 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":89,"threads_started":1}
I20260807 22:46:32.503075 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:32.502655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:47277)
0807 22:46:32.502905 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:32.502960 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":83,"threads_started":1}
I20260807 22:46:32.631184 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:32.630201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:50015)
0807 22:46:32.631023 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:32.631087 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":654,"threads_started":1}
I20260807 22:46:32.644687 21415 raft_consensus.cc:482] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:32.644814 21415 raft_consensus.cc:504] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a3866ecf4247b2901b6da5d14fb594" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 38231 } } peers { permanent_uuid: "9d708dc6518c4faabd162b92c4a59d12" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33071 } } peers { permanent_uuid: "df99f144f71040828f099bc42ff437e3" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42647 } }
I20260807 22:46:32.645177 21415 leader_election.cc:288] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231), 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071)
W20260807 22:46:32.645941 20802 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9d708dc6518c4faabd162b92c4a59d12 (127.26.189.194:33071): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
W20260807 22:46:32.646085 20801 leader_election.cc:341] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4a3866ecf4247b2901b6da5d14fb594 (127.26.189.195:38231): Not found: Tablet not found: 0bef3e38d0b144b3a4f539af3a4a9051
I20260807 22:46:32.646190 20801 leader_election.cc:302] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df99f144f71040828f099bc42ff437e3; no voters: 9d708dc6518c4faabd162b92c4a59d12, b4a3866ecf4247b2901b6da5d14fb594
I20260807 22:46:32.646399 21415 raft_consensus.cc:2726] T 0bef3e38d0b144b3a4f539af3a4a9051 P df99f144f71040828f099bc42ff437e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:46:32.813484 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:32.812997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:43037)
0807 22:46:32.813241 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:32.813365 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":94,"threads_started":1}
I20260807 22:46:33.504251 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:33.503827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:39297)
0807 22:46:33.504092 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:33.504152 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260807 22:46:33.632297 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:33.631903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:41009)
0807 22:46:33.632147 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:33.632204 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260807 22:46:33.819552 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:33.819099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:40957)
0807 22:46:33.819366 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:33.819430 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":76,"threads_started":1}
I20260807 22:46:34.505424 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.505005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:35909)
0807 22:46:34.505265 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:34.505327 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260807 22:46:34.633378 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.632974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:55665)
0807 22:46:34.633228 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:34.633286 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260807 22:46:34.820734 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:34.820249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:46049)
0807 22:46:34.820549 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:34.820615 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":78,"threads_started":1}
I20260807 22:46:35.507236 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:35.506619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.193:58009)
0807 22:46:35.506992 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:35.507089 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":113,"threads_started":1}
I20260807 22:46:35.635138 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:35.634561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.194:36507)
0807 22:46:35.634903 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:35.634978 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":97,"threads_started":1}
I20260807 22:46:35.802470 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 20710
I20260807 22:46:35.817157 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 20971
I20260807 22:46:35.823184 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:35.821833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38543 (local address 127.26.189.195:49049)
0807 22:46:35.822994 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:35.823072 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.254:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":75,"threads_started":1}
I20260807 22:46:35.827869 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 21156
2026-08-07T22:46:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59155 ms)
[----------] 30 tests from SecurityITest (238840 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:35 dist-test-slave-4tdj krb5kdc[21432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:35 dist-test-slave-4tdj krb5kdc[21432](info): set up 2 sockets
Aug 07 22:46:35 dist-test-slave-4tdj krb5kdc[21432](info): commencing operation
krb5kdc: starting...
W20260807 22:46:36.965595 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:46:37.086198 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43963
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:37.194906 21453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:37.195151 21453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:37.195206 21453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:37.199110 21453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:37.199163 21453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:37.199189 21453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:37.199209 21453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:37.199229 21453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:37.204286 21453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:37.205284 21453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:37.206254 21453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:37.214964 21461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.215227 21464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.215102 21453 server_base.cc:1034] running on GCE node
W20260807 22:46:37.214957 21460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.215858 21453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:37.216934 21453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:37.218106 21453 hybrid_clock.cc:648] HybridClock initialized: now 1786142797218082 us; error 36 us; skew 500 ppm
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:37.223845 21453 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:46:37.224720 21453 webserver.cc:460] Webserver started at http://127.26.189.254:41857/ using document root <none> and password file <none>
I20260807 22:46:37.225258 21453 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:37.225325 21453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:37.225531 21453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:37.227524 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "daaabc5e955c4ff9ba4384b5e51a00c8"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "43efc182bec0b003a32ce52caeeb7454"
server_key_iv: "fc72ad6391d6f1da03a8e16ffa003a29"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.228082 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "daaabc5e955c4ff9ba4384b5e51a00c8"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "43efc182bec0b003a32ce52caeeb7454"
server_key_iv: "fc72ad6391d6f1da03a8e16ffa003a29"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.231655 21453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:46:37.233873 21477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.235087 21453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:46:37.235208 21453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "daaabc5e955c4ff9ba4384b5e51a00c8"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "43efc182bec0b003a32ce52caeeb7454"
server_key_iv: "fc72ad6391d6f1da03a8e16ffa003a29"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.235308 21453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:37.249064 21453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:37.251691 21453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:37.251859 21453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:37.258739 21453 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43963
I20260807 22:46:37.258754 21551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43963 every 8 connection(s)
I20260807 22:46:37.259635 21453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:37.261937 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 21453
I20260807 22:46:37.262070 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:37.262382 21552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:37.262413 27383 external_mini_cluster.cc:1428] Setting key 69c5eba894ea9a298906cf0684c15e7e
I20260807 22:46:37.269261 21552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8: Bootstrap starting.
I20260807 22:46:37.271531 21552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8: Neither blocks nor log segments found. Creating new log.
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:37.272374 21552 log.cc:824] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:37.274406 21552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8: No bootstrap required, opened a new log
I20260807 22:46:37.276630 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.263770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.26.189.254:43963)
0807 22:46:37.264187 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:37.264206 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:37.264250 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:46:37.264988 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:37.265012 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:37.265433 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:37.265792 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:37.266545 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.267997 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.268847 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.269509 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.269682 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.272275 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:37.272280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:37.272293 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:37.272321 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:37.273995 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.274484 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.274491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.274497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.274626 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.274903 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.274905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.274906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.275281 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:37.275424 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:37.275689 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 22:46:37.275934 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
I20260807 22:46:37.277997 21552 raft_consensus.cc:348] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43963 } }
I20260807 22:46:37.278203 21552 raft_consensus.cc:374] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:37.278247 21552 raft_consensus.cc:729] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daaabc5e955c4ff9ba4384b5e51a00c8, State: Initialized, Role: FOLLOWER
I20260807 22:46:37.278663 21552 consensus_queue.cc:260] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43963 } }
I20260807 22:46:37.278812 21552 raft_consensus.cc:388] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:37.278875 21552 raft_consensus.cc:482] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:37.278966 21552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:37.279763 21552 raft_consensus.cc:504] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43963 } }
I20260807 22:46:37.280076 21552 leader_election.cc:302] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daaabc5e955c4ff9ba4384b5e51a00c8; no voters:
I20260807 22:46:37.280340 21552 leader_election.cc:288] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:37.280442 21557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:37.280665 21557 raft_consensus.cc:686] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [term 1 LEADER]: Becoming Leader. State: Replica: daaabc5e955c4ff9ba4384b5e51a00c8, State: Running, Role: LEADER
I20260807 22:46:37.280983 21557 consensus_queue.cc:237] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43963 } }
I20260807 22:46:37.281512 21552 sys_catalog.cc:564] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:37.282601 21562 sys_catalog.cc:455] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daaabc5e955c4ff9ba4384b5e51a00c8. Latest consensus state: current_term: 1 leader_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43963 } } }
I20260807 22:46:37.282716 21562 sys_catalog.cc:458] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:37.283015 21560 sys_catalog.cc:455] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daaabc5e955c4ff9ba4384b5e51a00c8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43963 } } }
I20260807 22:46:37.283092 21560 sys_catalog.cc:458] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:37.283097 21569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:37.285902 21569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:37.291105 21569 catalog_manager.cc:1347] Generated new cluster ID: 8ff4057083654fd8ac4fb5e08cbdd704
I20260807 22:46:37.291167 21569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:37.308156 21569 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:37.309026 21569 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:37.316774 21569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8: Generated new TSK 0
I20260807 22:46:37.317404 21569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:46:37.350842 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43963
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:37.459336 21584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:37.459566 21584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:37.459612 21584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:37.463434 21584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:37.463480 21584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:37.463533 21584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:37.468492 21584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:37.469386 21584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:37.470317 21584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:37.480109 21596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.480105 21594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.480162 21584 server_base.cc:1034] running on GCE node
W20260807 22:46:37.480105 21600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.480736 21584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:37.481284 21584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:37.482422 21584 hybrid_clock.cc:648] HybridClock initialized: now 1786142797482395 us; error 38 us; skew 500 ppm
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:37.484987 21584 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:46:37.485807 21584 webserver.cc:460] Webserver started at http://127.26.189.193:44621/ using document root <none> and password file <none>
I20260807 22:46:37.486349 21584 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:37.486418 21584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:37.486629 21584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:37.488546 21584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "2954930138164418ada2fe71c869bd35"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "51d26615f3b74cc063eca81323f6f0a7"
server_key_iv: "1b2fadc975e358835ff9c74f98d6d8d6"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.489075 21584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "2954930138164418ada2fe71c869bd35"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "51d26615f3b74cc063eca81323f6f0a7"
server_key_iv: "1b2fadc975e358835ff9c74f98d6d8d6"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.492552 21584 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 22:46:37.494805 21611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.495929 21584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:37.496029 21584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "2954930138164418ada2fe71c869bd35"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "51d26615f3b74cc063eca81323f6f0a7"
server_key_iv: "1b2fadc975e358835ff9c74f98d6d8d6"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.496116 21584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:37.519245 21584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:37.522234 21584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:37.522430 21584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:37.523003 21584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:37.523813 21584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:37.523859 21584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.523910 21584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:37.523939 21584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.532407 21584 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:34967
I20260807 22:46:37.532440 21763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:34967 every 8 connection(s)
I20260807 22:46:37.533228 21584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:37.537742 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 21584
I20260807 22:46:37.537863 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:37.538180 27383 external_mini_cluster.cc:1428] Setting key 7bf84c3fd99d66ea49c6823909dcda8d
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:37.546157 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.534961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:50191 (local address 127.26.189.254:43963)
0807 22:46:37.535083 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:37.535086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:37.535854 (+ 768us) server_negotiation.cc:407] Connection header received
0807 22:46:37.537112 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:37.537115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:37.537162 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:37.537230 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:37.538517 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.539070 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.540285 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.540522 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.540620 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.542957 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:37.542960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:37.542962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:37.542985 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:37.544265 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.544737 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.544741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.544742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.544815 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.545119 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.545122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.545124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.545307 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:37.545390 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:37.545867 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 22:46:37.545988 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:46:37.546917 21766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.535275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43963 (local address 127.26.189.193:50191)
0807 22:46:37.535698 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:37.535741 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:37.536923 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:37.537375 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:37.537381 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:37.537746 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:37.538367 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.538384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.539208 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.539211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.540164 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.540175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.540670 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.540674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.540841 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.541356 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:37.541371 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:37.542803 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:37.544373 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.544379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.544392 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.544645 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.544897 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.544900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.544902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.545032 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.545409 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:37.545415 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:37.545649 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 22:46:37.545805 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 22:46:37.546101 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:46:37.548087 21764 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43963
I20260807 22:46:37.548316 21764 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:37.548796 21764 heartbeater.cc:511] Master 127.26.189.254:43963 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
I20260807 22:46:37.550221 21509 ts_manager.cc:194] Registered new tserver with Master: 2954930138164418ada2fe71c869bd35 (127.26.189.193:34967)
I20260807 22:46:37.551388 21509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:50191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:46:37.564680 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43963
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:37.674878 21769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:37.675120 21769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:37.675163 21769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:37.679037 21769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:37.679086 21769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:37.679152 21769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:37.684248 21769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:37.685175 21769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:37.686146 21769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:37.695356 21787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.695389 21790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.695393 21786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.695530 21769 server_base.cc:1034] running on GCE node
I20260807 22:46:37.695979 21769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:37.696553 21769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:37.697721 21769 hybrid_clock.cc:648] HybridClock initialized: now 1786142797697705 us; error 31 us; skew 500 ppm
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:37.700417 21769 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:46:37.701260 21769 webserver.cc:460] Webserver started at http://127.26.189.194:36181/ using document root <none> and password file <none>
I20260807 22:46:37.701774 21769 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:37.701841 21769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:37.702033 21769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:37.704017 21769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "c49c7a6f4ce46fda00c12ea473d10b42"
server_key_iv: "b48c7d217d63d77bc4341c6471f7a5f8"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.704546 21769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "c49c7a6f4ce46fda00c12ea473d10b42"
server_key_iv: "b48c7d217d63d77bc4341c6471f7a5f8"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.708119 21769 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:46:37.710517 21800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.711735 21769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:37.711844 21769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "c49c7a6f4ce46fda00c12ea473d10b42"
server_key_iv: "b48c7d217d63d77bc4341c6471f7a5f8"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.711938 21769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:37.752154 21769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:37.755170 21769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:37.755357 21769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:37.755959 21769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:37.756796 21769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:37.756855 21769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.756910 21769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:37.756934 21769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.765522 21769 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:36519
I20260807 22:46:37.765604 21956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:36519 every 8 connection(s)
I20260807 22:46:37.766431 21769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:37.770946 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 21769
I20260807 22:46:37.771083 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:37.771368 27383 external_mini_cluster.cc:1428] Setting key eeb6504566ce45f02aeb048e59fb2168
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:37.779316 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.768121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:59917 (local address 127.26.189.254:43963)
0807 22:46:37.768262 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:37.768266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:37.769081 (+ 815us) server_negotiation.cc:407] Connection header received
0807 22:46:37.770342 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:37.770346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:37.770403 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:37.770478 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:37.771676 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.772280 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.773372 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.773591 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.773676 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.776144 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:37.776148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:37.776150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:37.776171 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:37.777409 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.777898 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.777901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.777902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.777976 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.778287 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.778315 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.778317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.778493 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:37.778579 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:37.779044 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 22:46:37.779162 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 22:46:37.780055 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.768475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43963 (local address 127.26.189.194:59917)
0807 22:46:37.768923 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:37.768968 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:37.770118 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:37.770655 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:37.770661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:37.771022 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:37.771540 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.771555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.772398 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.772401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.773257 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.773266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.773726 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.773731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.773883 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.774421 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:37.774437 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:37.775991 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:37.777557 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.777564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.777575 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.777805 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.778065 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.778068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.778070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.778200 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.778591 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:37.778597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:37.778820 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 22:46:37.778978 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 22:46:37.779274 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
I20260807 22:46:37.781206 21957 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43963
I20260807 22:46:37.781438 21957 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:37.781873 21957 heartbeater.cc:511] Master 127.26.189.254:43963 requested a full tablet report, sending...
I20260807 22:46:37.783034 21509 ts_manager.cc:194] Registered new tserver with Master: c11e7fc143b14dcdbecb9a8bdb9a4975 (127.26.189.194:36519)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
I20260807 22:46:37.783751 21509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:59917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:46:37.796880 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43963
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:37.905977 21963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:37.906217 21963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:37.906275 21963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:37.910251 21963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:37.910326 21963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:37.910400 21963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:37.915618 21963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:43963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:37.916620 21963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:37.917595 21963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:37.927089 21972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.927102 21968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:37.927099 21969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:37.927212 21963 server_base.cc:1034] running on GCE node
I20260807 22:46:37.927670 21963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:37.928283 21963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:37.929440 21963 hybrid_clock.cc:648] HybridClock initialized: now 1786142797929423 us; error 31 us; skew 500 ppm
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:37.932160 21963 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:46:37.932978 21963 webserver.cc:460] Webserver started at http://127.26.189.195:33517/ using document root <none> and password file <none>
I20260807 22:46:37.933503 21963 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:37.933568 21963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:37.933760 21963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:37.935778 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "d4d06e4fbf424814b3999d2e4c258777"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "571efb89b06ac671455da0994056a924"
server_key_iv: "889d8b4a4e5364abf2ca9d8e0da57e8b"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.936327 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "d4d06e4fbf424814b3999d2e4c258777"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "571efb89b06ac671455da0994056a924"
server_key_iv: "889d8b4a4e5364abf2ca9d8e0da57e8b"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.940007 21963 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:46:37.942370 21980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.943563 21963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:37.943673 21963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "d4d06e4fbf424814b3999d2e4c258777"
format_stamp: "Formatted at 2026-08-07 22:46:37 on dist-test-slave-4tdj"
server_key: "571efb89b06ac671455da0994056a924"
server_key_iv: "889d8b4a4e5364abf2ca9d8e0da57e8b"
server_key_version: "encryptionkey@0"
I20260807 22:46:37.943773 21963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:37.965929 21963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:37.969017 21963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:37.969194 21963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:37.969763 21963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:37.970582 21963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:37.970628 21963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.970669 21963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:37.970698 21963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:37.979141 21963 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:36961
I20260807 22:46:37.979169 22132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:36961 every 8 connection(s)
I20260807 22:46:37.979936 21963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:37.983047 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 21963
I20260807 22:46:37.983201 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:37.983554 27383 external_mini_cluster.cc:1428] Setting key 7d34d1a39a40ec5b6f778ab36a7c830e
Aug 07 22:46:37 dist-test-slave-4tdj krb5kdc[21432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142797, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:37.992797 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.981609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:33631 (local address 127.26.189.254:43963)
0807 22:46:37.981748 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:37.981753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:37.982581 (+ 828us) server_negotiation.cc:407] Connection header received
0807 22:46:37.984273 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:37.984277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:37.984367 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:37.984476 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:37.985888 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.986451 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.987491 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.987702 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.987771 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.989902 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:37.989905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:37.989907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:37.989928 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:37.991222 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.991661 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.991664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.991665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.991732 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:37.991976 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:37.991978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:37.991979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:37.992131 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:37.992197 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:37.992598 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 22:46:37.992684 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:46:37.993539 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.981989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:43963 (local address 127.26.189.195:33631)
0807 22:46:37.982439 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:37.982480 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:37.983996 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:37.984871 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:37.984879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:37.985252 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:37.985766 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.985780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.986550 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.986553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.987402 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:37.987411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:37.987813 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:37.987815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:37.987953 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:37.988402 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:37.988417 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:37.989756 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:37.991323 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.991328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.991338 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.991583 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.991792 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:37.991794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:37.991795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:37.991914 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:37.992215 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:37.992219 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:37.992424 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 22:46:37.992570 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 22:46:37.992795 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":118,"threads_started":1}
I20260807 22:46:37.994612 22134 heartbeater.cc:347] Connected to a master server at 127.26.189.254:43963
I20260807 22:46:37.994848 22134 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:37.995292 22134 heartbeater.cc:511] Master 127.26.189.254:43963 requested a full tablet report, sending...
I20260807 22:46:37.996290 21509 ts_manager.cc:194] Registered new tserver with Master: d4d06e4fbf424814b3999d2e4c258777 (127.26.189.195:36961)
I20260807 22:46:37.996902 21509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:33631
I20260807 22:46:37.997231 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:38.006179 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:37.999348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.26.189.254:43963)
0807 22:46:37.999491 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:37.999495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:37.999720 (+ 225us) server_negotiation.cc:407] Connection header received
0807 22:46:37.999821 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:37.999824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:37.999863 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:37.999925 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.000646 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.001135 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.001922 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.002145 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.002226 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.003359 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.003362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.003365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:38.003390 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.004626 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.004990 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.004993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.004995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.005076 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:38.005332 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:38.005336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:38.005337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:38.005534 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.005648 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.005874 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.006024 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:46:38.008486 21509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:38.011422 21509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:38.025107 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.019158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:36519 (local address 127.0.0.1:53288)
0807 22:46:38.019490 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.019541 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.019793 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.021009 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.021015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.021044 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.021163 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.021170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.022184 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.022188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.023432 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.023444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.024545 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.024549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.024669 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.024712 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.024824 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.024942 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":27,"thread_start_us":69,"threads_started":1}
I20260807 22:46:38.025254 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.019543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34967 (local address 127.0.0.1:34924)
0807 22:46:38.020076 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.020089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.020172 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.021273 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.021276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.021294 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.021375 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.021380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.022838 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.022841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.023875 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.023884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.024821 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.024825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.024913 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.024936 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.025021 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.025079 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":36,"thread_start_us":100,"threads_started":1}
I20260807 22:46:38.025583 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.019369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.26.189.194:36519)
0807 22:46:38.020552 (+ 1183us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.020565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.020587 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:46:38.020653 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.020657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.020841 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.021007 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.021274 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.022061 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.023565 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.024426 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.024547 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.024684 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.025320 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.025431 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1062,"thread_start_us":118,"threads_started":1}
I20260807 22:46:38.025820 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.020268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34924 (local address 127.26.189.193:34967)
0807 22:46:38.020561 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.020575 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.020598 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:46:38.020674 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.020680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.020861 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.021004 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.021496 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.022670 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.023988 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.024661 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.024811 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.024926 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.025064 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.025159 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":106,"threads_started":1}
I20260807 22:46:38.025949 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.020021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.26.189.195:36961)
0807 22:46:38.020389 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.020401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.020740 (+ 339us) server_negotiation.cc:407] Connection header received
0807 22:46:38.020818 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.020823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.020989 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.021126 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.021434 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.022802 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.024381 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.025159 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.025292 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.025409 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.025742 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.025830 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":144,"threads_started":1}
I20260807 22:46:38.026226 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.019861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36961 (local address 127.0.0.1:50510)
0807 22:46:38.020634 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.020646 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.020736 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.021195 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.021198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.021217 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.021322 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.021327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.022943 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.022946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.024231 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.024244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.025471 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.025474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.025573 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.025595 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.026067 (+ 472us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.026141 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":85,"threads_started":1}
I20260807 22:46:38.028209 21862 tablet_service.cc:1467] Processing CreateTablet for tablet dc20afd327d448559bb4706093d25f0e (DEFAULT_TABLE table=test-table [id=393769f9cdb746fab89aa380b6fe1a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:38.028206 21672 tablet_service.cc:1467] Processing CreateTablet for tablet dc20afd327d448559bb4706093d25f0e (DEFAULT_TABLE table=test-table [id=393769f9cdb746fab89aa380b6fe1a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:38.029102 21672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc20afd327d448559bb4706093d25f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.029102 21862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc20afd327d448559bb4706093d25f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.032416 22042 tablet_service.cc:1467] Processing CreateTablet for tablet dc20afd327d448559bb4706093d25f0e (DEFAULT_TABLE table=test-table [id=393769f9cdb746fab89aa380b6fe1a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:38.033365 22042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc20afd327d448559bb4706093d25f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.035293 22154 tablet_bootstrap.cc:492] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Bootstrap starting.
I20260807 22:46:38.036080 22155 tablet_bootstrap.cc:492] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: Bootstrap starting.
I20260807 22:46:38.037679 22154 tablet_bootstrap.cc:654] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.038398 22155 tablet_bootstrap.cc:654] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.039116 22154 log.cc:824] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.039640 22155 log.cc:824] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.041252 22154 tablet_bootstrap.cc:492] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: No bootstrap required, opened a new log
I20260807 22:46:38.039942 22156 tablet_bootstrap.cc:492] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Bootstrap starting.
I20260807 22:46:38.041462 22154 ts_tablet_manager.cc:1397] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.006s
I20260807 22:46:38.041647 22155 tablet_bootstrap.cc:492] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: No bootstrap required, opened a new log
I20260807 22:46:38.041839 22155 ts_tablet_manager.cc:1397] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 22:46:38.043156 22156 tablet_bootstrap.cc:654] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.044093 22156 log.cc:824] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.045609 22154 raft_consensus.cc:348] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.045674 22155 raft_consensus.cc:348] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.045876 22156 tablet_bootstrap.cc:492] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: No bootstrap required, opened a new log
I20260807 22:46:38.045902 22155 raft_consensus.cc:374] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.045902 22154 raft_consensus.cc:374] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.045979 22154 raft_consensus.cc:729] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c11e7fc143b14dcdbecb9a8bdb9a4975, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.045997 22155 raft_consensus.cc:729] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2954930138164418ada2fe71c869bd35, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.046056 22156 ts_tablet_manager.cc:1397] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 22:46:38.046599 22155 consensus_queue.cc:260] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.046599 22154 consensus_queue.cc:260] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.047547 21957 heartbeater.cc:503] Master 127.26.189.254:43963 was elected leader, sending a full tablet report...
I20260807 22:46:38.047696 22154 ts_tablet_manager.cc:1428] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 22:46:38.047696 22155 ts_tablet_manager.cc:1428] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 22:46:38.048418 21764 heartbeater.cc:503] Master 127.26.189.254:43963 was elected leader, sending a full tablet report...
I20260807 22:46:38.049502 22156 raft_consensus.cc:348] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.049778 22156 raft_consensus.cc:374] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.049849 22156 raft_consensus.cc:729] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d06e4fbf424814b3999d2e4c258777, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.050324 22156 consensus_queue.cc:260] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.051302 22134 heartbeater.cc:503] Master 127.26.189.254:43963 was elected leader, sending a full tablet report...
I20260807 22:46:38.051366 22156 ts_tablet_manager.cc:1428] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:46:38.196139 22162 raft_consensus.cc:482] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:38.196365 22162 raft_consensus.cc:504] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.197507 22162 leader_election.cc:288] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c11e7fc143b14dcdbecb9a8bdb9a4975 (127.26.189.194:36519), d4d06e4fbf424814b3999d2e4c258777 (127.26.189.195:36961)
I20260807 22:46:38.202044 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.197892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:56573 (local address 127.26.189.194:36519)
0807 22:46:38.198076 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.198082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.198212 (+ 130us) server_negotiation.cc:407] Connection header received
0807 22:46:38.198464 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.198467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.198553 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.198614 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.199056 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.199573 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.200947 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.201538 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.201653 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.201697 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.201874 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.201943 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:46:38.202075 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.197806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:36519 (local address 127.26.189.193:56573)
0807 22:46:38.198131 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.198147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.198323 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.198783 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.198804 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.198825 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.198939 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.198945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.199655 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.199658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.200825 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.200835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.201650 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.201654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.201750 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.201776 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.201847 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.201914 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
I20260807 22:46:38.202736 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.198961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:38033 (local address 127.26.189.195:36961)
0807 22:46:38.199090 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.199095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.199108 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:46:38.199158 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.199160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.199201 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.199264 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:38.199510 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.200319 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.201582 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.202145 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.202255 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.202326 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.202550 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.202623 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:46:38.202759 22166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.198239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36961 (local address 127.26.189.193:38033)
0807 22:46:38.198663 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.198677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.198799 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.199284 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.199287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.199305 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:38.199410 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.199417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.200448 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.200451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.201450 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.201460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.202317 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.202321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.202416 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.202440 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.202513 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.202622 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":128,"threads_started":1}
I20260807 22:46:38.202752 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc20afd327d448559bb4706093d25f0e" candidate_uuid: "2954930138164418ada2fe71c869bd35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" is_pre_election: true
I20260807 22:46:38.203080 21890 raft_consensus.cc:2454] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2954930138164418ada2fe71c869bd35 in term 0.
I20260807 22:46:38.203301 22073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc20afd327d448559bb4706093d25f0e" candidate_uuid: "2954930138164418ada2fe71c869bd35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d06e4fbf424814b3999d2e4c258777" is_pre_election: true
I20260807 22:46:38.203567 22073 raft_consensus.cc:2454] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2954930138164418ada2fe71c869bd35 in term 0.
I20260807 22:46:38.203553 21614 leader_election.cc:302] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2954930138164418ada2fe71c869bd35, c11e7fc143b14dcdbecb9a8bdb9a4975; no voters:
I20260807 22:46:38.203781 22162 raft_consensus.cc:2781] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:38.203869 22162 raft_consensus.cc:482] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:38.203920 22162 raft_consensus.cc:3037] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:38.204957 22162 raft_consensus.cc:504] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.205273 22162 leader_election.cc:288] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [CANDIDATE]: Term 1 election: Requested vote from peers c11e7fc143b14dcdbecb9a8bdb9a4975 (127.26.189.194:36519), d4d06e4fbf424814b3999d2e4c258777 (127.26.189.195:36961)
I20260807 22:46:38.205613 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc20afd327d448559bb4706093d25f0e" candidate_uuid: "2954930138164418ada2fe71c869bd35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975"
I20260807 22:46:38.205632 22073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc20afd327d448559bb4706093d25f0e" candidate_uuid: "2954930138164418ada2fe71c869bd35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d06e4fbf424814b3999d2e4c258777"
I20260807 22:46:38.205739 22073 raft_consensus.cc:3037] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:38.205739 21890 raft_consensus.cc:3037] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:38.207044 22073 raft_consensus.cc:2454] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2954930138164418ada2fe71c869bd35 in term 1.
I20260807 22:46:38.207070 21890 raft_consensus.cc:2454] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2954930138164418ada2fe71c869bd35 in term 1.
I20260807 22:46:38.207343 21614 leader_election.cc:302] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2954930138164418ada2fe71c869bd35, d4d06e4fbf424814b3999d2e4c258777; no voters:
I20260807 22:46:38.207497 22162 raft_consensus.cc:2781] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:38.207728 22162 raft_consensus.cc:686] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 1 LEADER]: Becoming Leader. State: Replica: 2954930138164418ada2fe71c869bd35, State: Running, Role: LEADER
I20260807 22:46:38.208024 22162 consensus_queue.cc:237] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } }
I20260807 22:46:38.211333 21508 catalog_manager.cc:5495] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2954930138164418ada2fe71c869bd35 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "2954930138164418ada2fe71c869bd35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2954930138164418ada2fe71c869bd35" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 34967 } health_report { overall_health: HEALTHY } } }
W20260807 22:46:38.230854 22135 tablet.cc:2369] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:38.263089 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.259410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.26.189.193:34967)
0807 22:46:38.259566 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.259571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.259588 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:46:38.259673 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.259676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.259740 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.259834 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:38.260044 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.260756 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.261547 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.261890 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.261985 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.262251 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:38.262734 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:38.262839 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.262891 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.262949 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 22:46:38.267539 21958 tablet.cc:2369] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:38.269028 21890 raft_consensus.cc:1261] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 1 FOLLOWER]: Refusing update from remote peer 2954930138164418ada2fe71c869bd35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:38.269057 22073 raft_consensus.cc:1261] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 1 FOLLOWER]: Refusing update from remote peer 2954930138164418ada2fe71c869bd35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:38.269706 22162 consensus_queue.cc:1035] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d06e4fbf424814b3999d2e4c258777" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:38.269918 22168 consensus_queue.cc:1035] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11e7fc143b14dcdbecb9a8bdb9a4975" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 36519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:38.280205 22178 mvcc.cc:205] Tried to move back new op lower bound from 7316040901703172096 to 7316040901462896640. Current Snapshot: MvccSnapshot[applied={T|T < 7316040901703172096}]
I20260807 22:46:38.284415 21509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:38.284582 21509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:38.287297 21509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P daaabc5e955c4ff9ba4384b5e51a00c8: Sending DeleteTablet for 3 replicas of tablet dc20afd327d448559bb4706093d25f0e
I20260807 22:46:38.288081 21672 tablet_service.cc:1514] Processing DeleteTablet for tablet dc20afd327d448559bb4706093d25f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:34924
I20260807 22:46:38.288208 21862 tablet_service.cc:1514] Processing DeleteTablet for tablet dc20afd327d448559bb4706093d25f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53288
I20260807 22:46:38.288579 22189 tablet_replica.cc:330] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35: stopping tablet replica
I20260807 22:46:38.288602 22190 tablet_replica.cc:330] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: stopping tablet replica
I20260807 22:46:38.288632 22042 tablet_service.cc:1514] Processing DeleteTablet for tablet dc20afd327d448559bb4706093d25f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:38 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:50510
I20260807 22:46:38.288897 22190 raft_consensus.cc:2229] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:38.288899 22189 raft_consensus.cc:2229] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:38.289242 22189 raft_consensus.cc:2258] T dc20afd327d448559bb4706093d25f0e P 2954930138164418ada2fe71c869bd35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:38.289249 22190 raft_consensus.cc:2258] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:38.289611 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 21584
I20260807 22:46:38.290853 22190 ts_tablet_manager.cc:1905] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:38.293114 22190 ts_tablet_manager.cc:1918] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:38.293200 22190 log.cc:1196] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/dc20afd327d448559bb4706093d25f0e
I20260807 22:46:38.293479 22190 ts_tablet_manager.cc:1939] T dc20afd327d448559bb4706093d25f0e P c11e7fc143b14dcdbecb9a8bdb9a4975: Deleting consensus metadata
I20260807 22:46:38.296411 21479 catalog_manager.cc:4841] TS c11e7fc143b14dcdbecb9a8bdb9a4975 (127.26.189.194:36519): tablet dc20afd327d448559bb4706093d25f0e (table test-table [id=393769f9cdb746fab89aa380b6fe1a57]) successfully deleted
I20260807 22:46:38.298029 22191 tablet_replica.cc:330] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: stopping tablet replica
I20260807 22:46:38.298384 22191 raft_consensus.cc:2229] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:38.298715 22191 raft_consensus.cc:2258] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:38.300033 22191 ts_tablet_manager.cc:1905] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:38.302114 22191 ts_tablet_manager.cc:1918] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:38.302203 22191 log.cc:1196] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/dc20afd327d448559bb4706093d25f0e
I20260807 22:46:38.302521 22191 ts_tablet_manager.cc:1939] T dc20afd327d448559bb4706093d25f0e P d4d06e4fbf424814b3999d2e4c258777: Deleting consensus metadata
W20260807 22:46:38.303653 21478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:34967 (error 108)
I20260807 22:46:38.303674 21479 catalog_manager.cc:4841] TS d4d06e4fbf424814b3999d2e4c258777 (127.26.189.195:36961): tablet dc20afd327d448559bb4706093d25f0e (table test-table [id=393769f9cdb746fab89aa380b6fe1a57]) successfully deleted
I20260807 22:46:38.303844 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 21769
I20260807 22:46:38.309154 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.306736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:34967 (local address 127.0.0.1:34948)
0807 22:46:38.309011 (+ 2275us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.309079 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2192}
W20260807 22:46:38.309324 21478 catalog_manager.cc:4584] TS 2954930138164418ada2fe71c869bd35 (127.26.189.193:34967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc20afd327d448559bb4706093d25f0e: Network error: Client connection negotiation failed: client connection to 127.26.189.193:34967: connect: Connection refused (error 111)
I20260807 22:46:38.315263 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 21963
I20260807 22:46:38.328053 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 21453
2026-08-07T22:46:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2498 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T22:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:38Z Disabled control of system clock
I20260807 22:46:38.361871 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38713
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:46:38.471007 22197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:38.471263 22197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:38.471297 22197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:38.475400 22197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:38.475473 22197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:38.475494 22197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:38.475514 22197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:38.475533 22197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:38.481035 22197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:38.482262 22197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:38.483736 22197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:38.491920 22205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.491909 22209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.491920 22206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:38.492273 22197 server_base.cc:1034] running on GCE node
I20260807 22:46:38.492808 22197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:38.493937 22197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:38.495111 22197 hybrid_clock.cc:648] HybridClock initialized: now 1786142798495090 us; error 37 us; skew 500 ppm
I20260807 22:46:38.497005 22197 webserver.cc:460] Webserver started at http://127.26.189.254:39727/ using document root <none> and password file <none>
I20260807 22:46:38.497542 22197 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:38.497602 22197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:38.497763 22197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:38.499820 22197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "b3af48b8187d3541c9ca05f775b0eb20"
server_key_iv: "17a1c22d4f271095a1e86e4e52cd7ed7"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.500568 22197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "b3af48b8187d3541c9ca05f775b0eb20"
server_key_iv: "17a1c22d4f271095a1e86e4e52cd7ed7"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.504269 22197 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:46:38.506878 22221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.508200 22197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:38.508307 22197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "b3af48b8187d3541c9ca05f775b0eb20"
server_key_iv: "17a1c22d4f271095a1e86e4e52cd7ed7"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.508397 22197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:38.527812 22197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:38.528491 22197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:38.528646 22197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:38.535797 22197 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38713
I20260807 22:46:38.535820 22290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38713 every 8 connection(s)
I20260807 22:46:38.536749 22197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:38.537705 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 22197
I20260807 22:46:38.537850 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:38.538168 27383 external_mini_cluster.cc:1428] Setting key 9985629232571f6be3e02fdd5f9ac10a
I20260807 22:46:38.540427 22291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:38.546501 22291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f: Bootstrap starting.
I20260807 22:46:38.547312 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.539687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.26.189.254:38713)
0807 22:46:38.540354 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.540369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.540405 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:46:38.541067 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.541085 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.541459 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.541743 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.542078 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.543070 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.544230 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.544668 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.544779 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.545651 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.545656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.545668 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:38.546030 (+ 362us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.546096 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:38.546115 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.546187 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.546365 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.546621 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":150,"threads_started":1}
I20260807 22:46:38.548846 22291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:38.549633 22291 log.cc:824] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:38.551470 22291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f: No bootstrap required, opened a new log
I20260807 22:46:38.554771 22291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38713 } }
I20260807 22:46:38.554982 22291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:38.555048 22291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b6dfaf5a0044cb8f70d8f75b9fb50f, State: Initialized, Role: FOLLOWER
I20260807 22:46:38.555480 22291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38713 } }
I20260807 22:46:38.555639 22291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:38.555707 22291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:38.555800 22291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:38.556655 22291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38713 } }
I20260807 22:46:38.557024 22291 leader_election.cc:302] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b6dfaf5a0044cb8f70d8f75b9fb50f; no voters:
I20260807 22:46:38.557343 22291 leader_election.cc:288] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:38.557440 22298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:38.557670 22298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [term 1 LEADER]: Becoming Leader. State: Replica: 55b6dfaf5a0044cb8f70d8f75b9fb50f, State: Running, Role: LEADER
I20260807 22:46:38.558004 22298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38713 } }
I20260807 22:46:38.558542 22291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:38.559574 22299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38713 } } }
I20260807 22:46:38.559698 22299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:38.559824 22300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b6dfaf5a0044cb8f70d8f75b9fb50f. Latest consensus state: current_term: 1 leader_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b6dfaf5a0044cb8f70d8f75b9fb50f" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38713 } } }
I20260807 22:46:38.559957 22300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:38.560132 22308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:38.562987 22308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:38.568397 22308 catalog_manager.cc:1347] Generated new cluster ID: 22406341f49044f0b44c3e18650ba762
I20260807 22:46:38.568459 22308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:38.585593 22308 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:38.586534 22308 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:38.596235 22308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f: Generated new TSK 0
I20260807 22:46:38.596869 22308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:38.606660 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38713
--builtin_ntp_servers=127.26.189.212:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:46:38.715502 22322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:38.715749 22322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:38.715785 22322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:38.719666 22322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:38.719710 22322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:38.719763 22322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:38.724375 22322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:38.725384 22322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:38.726714 22322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:38.735729 22331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.735736 22333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.735736 22330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:38.735801 22322 server_base.cc:1034] running on GCE node
I20260807 22:46:38.736305 22322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:38.736887 22322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:38.738058 22322 hybrid_clock.cc:648] HybridClock initialized: now 1786142798738033 us; error 38 us; skew 500 ppm
I20260807 22:46:38.739799 22322 webserver.cc:460] Webserver started at http://127.26.189.193:44433/ using document root <none> and password file <none>
I20260807 22:46:38.740346 22322 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:38.740413 22322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:38.740621 22322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:38.742719 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "0ce6452c942945df8b1aedc4e17f85d9"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "5fdbf42fb0e5ef1c7bab8d53512b1380"
server_key_iv: "9733a42024cb2f602f3f6e5cb3e62b9b"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.743247 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "0ce6452c942945df8b1aedc4e17f85d9"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "5fdbf42fb0e5ef1c7bab8d53512b1380"
server_key_iv: "9733a42024cb2f602f3f6e5cb3e62b9b"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.746688 22322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:46:38.748914 22342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.750057 22322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:38.750159 22322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "0ce6452c942945df8b1aedc4e17f85d9"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "5fdbf42fb0e5ef1c7bab8d53512b1380"
server_key_iv: "9733a42024cb2f602f3f6e5cb3e62b9b"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.750248 22322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:38.779831 22322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:38.780536 22322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:38.780711 22322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:38.781292 22322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:38.782228 22322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:38.782277 22322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.782348 22322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:38.782379 22322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.790735 22322 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42413
I20260807 22:46:38.790783 22500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42413 every 8 connection(s)
I20260807 22:46:38.791587 22322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:38.793366 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 22322
I20260807 22:46:38.793534 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:38.793824 27383 external_mini_cluster.cc:1428] Setting key 75f1de059acfc5365181a7797b0139aa
I20260807 22:46:38.795931 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38713
--builtin_ntp_servers=127.26.189.212:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:46:38.801352 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.793331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:56385 (local address 127.26.189.254:38713)
0807 22:46:38.793460 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.793464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.794758 (+ 1294us) server_negotiation.cc:407] Connection header received
0807 22:46:38.796288 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.796292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.796346 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.796429 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.797262 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.797911 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.799377 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.799689 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.799795 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.800851 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.800855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.800858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:38.800874 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.800898 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:38.800908 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.800997 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.801114 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.801184 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:46:38.802096 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.793827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38713 (local address 127.26.189.193:56385)
0807 22:46:38.794604 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.794660 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.796057 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.796589 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.796595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.796673 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:38.797128 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.797140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.798047 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.798051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.799247 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.799258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.799855 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.799858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.800010 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.800602 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:38.800604 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:38.800642 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:38.800703 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:38.800993 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:38.800998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:38.801001 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.801096 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.801305 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":155,"threads_started":1}
I20260807 22:46:38.803272 22501 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38713
I20260807 22:46:38.803522 22501 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:38.804152 22501 heartbeater.cc:511] Master 127.26.189.254:38713 requested a full tablet report, sending...
I20260807 22:46:38.805691 22244 ts_manager.cc:194] Registered new tserver with Master: 0ce6452c942945df8b1aedc4e17f85d9 (127.26.189.193:42413)
I20260807 22:46:38.806838 22244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.193:56385
W20260807 22:46:38.907341 22504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:38.907582 22504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:38.907618 22504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:38.911522 22504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:38.911569 22504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:38.911641 22504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:38.916395 22504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:38.917300 22504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:38.918627 22504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:38.927752 22510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.927750 22512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:38.927752 22509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:38.928083 22504 server_base.cc:1034] running on GCE node
I20260807 22:46:38.928412 22504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:38.928977 22504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:38.930127 22504 hybrid_clock.cc:648] HybridClock initialized: now 1786142798930108 us; error 36 us; skew 500 ppm
I20260807 22:46:38.931854 22504 webserver.cc:460] Webserver started at http://127.26.189.194:46767/ using document root <none> and password file <none>
I20260807 22:46:38.932427 22504 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:38.932497 22504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:38.932703 22504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:38.934669 22504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "baa01af0fba946858c8fd4779ccdd86e"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "c583faac7b1c4472522a3f343265136d"
server_key_iv: "560d8a3c5458bb4be933e63c2dd5b2a5"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.935186 22504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "baa01af0fba946858c8fd4779ccdd86e"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "c583faac7b1c4472522a3f343265136d"
server_key_iv: "560d8a3c5458bb4be933e63c2dd5b2a5"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.938683 22504 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:46:38.940968 22523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.942101 22504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:38.942320 22504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "baa01af0fba946858c8fd4779ccdd86e"
format_stamp: "Formatted at 2026-08-07 22:46:38 on dist-test-slave-4tdj"
server_key: "c583faac7b1c4472522a3f343265136d"
server_key_iv: "560d8a3c5458bb4be933e63c2dd5b2a5"
server_key_version: "encryptionkey@0"
I20260807 22:46:38.942436 22504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:38.969200 22504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:38.969847 22504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:38.970026 22504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:38.970628 22504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:38.971547 22504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:38.971596 22504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.971647 22504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:38.971675 22504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:38.979776 22504 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:45749
I20260807 22:46:38.979831 22682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:45749 every 8 connection(s)
I20260807 22:46:38.980648 22504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:38.983075 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 22504
I20260807 22:46:38.983182 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:38.983508 27383 external_mini_cluster.cc:1428] Setting key efa9d08651366e587800151e184f3947
I20260807 22:46:38.985699 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38713
--builtin_ntp_servers=127.26.189.212:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:46:38.990384 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.982492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:52989 (local address 127.26.189.254:38713)
0807 22:46:38.982670 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:38.982676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:38.983866 (+ 1190us) server_negotiation.cc:407] Connection header received
0807 22:46:38.985353 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:38.985356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:38.985401 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:38.985458 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:38.986599 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.987126 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.988371 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.988695 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.988804 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.989870 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:38.989875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:38.989878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:38.989893 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:38.989917 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:38.989930 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:38.990031 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:38.990138 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 22:46:38.990210 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 22:46:38.991222 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:38.982914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38713 (local address 127.26.189.194:52989)
0807 22:46:38.983692 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:38.983748 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:38.985109 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:38.985647 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:38.985656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:38.985762 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:38.986429 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.986448 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.987256 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.987259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.988234 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:38.988244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:38.988868 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:38.988871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:38.989033 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:38.989614 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:38.989616 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:38.989655 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:38.989716 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:38.990023 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:38.990028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:38.990031 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:46:38.990116 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 22:46:38.990394 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":127,"threads_started":1}
I20260807 22:46:38.992327 22684 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38713
I20260807 22:46:38.992553 22684 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:38.993055 22684 heartbeater.cc:511] Master 127.26.189.254:38713 requested a full tablet report, sending...
I20260807 22:46:38.994036 22244 ts_manager.cc:194] Registered new tserver with Master: baa01af0fba946858c8fd4779ccdd86e (127.26.189.194:45749)
I20260807 22:46:38.994663 22244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.194:52989
W20260807 22:46:39.098201 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:39.098485 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:39.098531 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:39.102415 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:39.102468 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:39.102521 22688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:39.107074 22688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:39.108018 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:39.109287 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:39.118569 22703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:39.118585 22698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:39.118765 22688 server_base.cc:1034] running on GCE node
W20260807 22:46:39.118647 22699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:39.119175 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:39.119745 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:39.120908 22688 hybrid_clock.cc:648] HybridClock initialized: now 1786142799120884 us; error 35 us; skew 500 ppm
I20260807 22:46:39.122738 22688 webserver.cc:460] Webserver started at http://127.26.189.195:34279/ using document root <none> and password file <none>
I20260807 22:46:39.123260 22688 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:39.123327 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:39.123533 22688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:39.125550 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "5942ec00bb7344a9bd3f035f043d3624"
format_stamp: "Formatted at 2026-08-07 22:46:39 on dist-test-slave-4tdj"
server_key: "88ecacafe1731ec7d3a29549992e433c"
server_key_iv: "0934858419285c2e6e2fa4d9fe9a6da6"
server_key_version: "encryptionkey@0"
I20260807 22:46:39.126103 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "5942ec00bb7344a9bd3f035f043d3624"
format_stamp: "Formatted at 2026-08-07 22:46:39 on dist-test-slave-4tdj"
server_key: "88ecacafe1731ec7d3a29549992e433c"
server_key_iv: "0934858419285c2e6e2fa4d9fe9a6da6"
server_key_version: "encryptionkey@0"
I20260807 22:46:39.129660 22688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:46:39.132015 22709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:39.133231 22688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 22:46:39.133335 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "5942ec00bb7344a9bd3f035f043d3624"
format_stamp: "Formatted at 2026-08-07 22:46:39 on dist-test-slave-4tdj"
server_key: "88ecacafe1731ec7d3a29549992e433c"
server_key_iv: "0934858419285c2e6e2fa4d9fe9a6da6"
server_key_version: "encryptionkey@0"
I20260807 22:46:39.133435 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:39.146456 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:39.147029 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:39.147179 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:39.147703 22688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:39.148576 22688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:39.148620 22688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:39.148660 22688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:39.148689 22688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:39.156955 22688 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:35223
I20260807 22:46:39.156978 22871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:35223 every 8 connection(s)
I20260807 22:46:39.157783 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:39.160022 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 22688
I20260807 22:46:39.160117 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:39.160365 27383 external_mini_cluster.cc:1428] Setting key a2c68685cb5934edf988bf63b3046916
I20260807 22:46:39.167052 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.159639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:35593 (local address 127.26.189.254:38713)
0807 22:46:39.159778 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.159783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.160965 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 22:46:39.162355 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.162359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.162422 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.162541 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:39.163408 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.163951 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.165103 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.165343 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.165441 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.166551 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:39.166556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:39.166559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:39.166573 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:39.166594 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:39.166603 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:39.166665 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.166818 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.166884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:46:39.167816 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.160017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38713 (local address 127.26.189.195:35593)
0807 22:46:39.160784 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.160845 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:39.162123 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:39.162708 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:39.162715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:39.162797 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:39.163258 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.163270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.164094 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.164097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.164972 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.164982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.165498 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.165503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.165669 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.166264 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:39.166266 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:39.166323 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:39.166386 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:39.166676 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:39.166685 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:39.166692 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 22:46:39.166809 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 22:46:39.167073 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":132,"threads_started":1}
I20260807 22:46:39.168769 22873 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38713
I20260807 22:46:39.168982 22873 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:39.169499 22873 heartbeater.cc:511] Master 127.26.189.254:38713 requested a full tablet report, sending...
I20260807 22:46:39.170565 22244 ts_manager.cc:194] Registered new tserver with Master: 5942ec00bb7344a9bd3f035f043d3624 (127.26.189.195:35223)
I20260807 22:46:39.171114 22244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.195:35593
I20260807 22:46:39.173851 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:39.179204 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.175819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55994 (local address 127.26.189.254:38713)
0807 22:46:39.175929 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.175932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.176152 (+ 220us) server_negotiation.cc:407] Connection header received
0807 22:46:39.176257 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.176259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.176293 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.176355 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:39.176689 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.177158 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.177923 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.178129 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.178196 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.178915 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:39.178918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:39.178919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:39.178929 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:39.178943 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:39.178950 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:39.179003 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.179071 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.179111 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:46:39.181488 22244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:39.184430 22244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:39.197813 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.192081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:35223 (local address 127.0.0.1:34468)
0807 22:46:39.192995 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.193047 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:39.193186 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:39.193824 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:39.193830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:39.193853 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:39.193991 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.193999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.195148 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.195152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.196499 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.196512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.197317 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.197320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.197422 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.197457 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:46:39.197569 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:46:39.197679 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":86,"threads_started":1}
I20260807 22:46:39.198110 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.192521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.26.189.193:42413)
0807 22:46:39.193020 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.193036 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.193335 (+ 299us) server_negotiation.cc:407] Connection header received
0807 22:46:39.193418 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.193424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.193628 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.193773 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:39.194181 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.195304 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.196666 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.197469 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.197611 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.197796 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.197849 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.197938 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":139,"threads_started":1}
I20260807 22:46:39.198392 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.192236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.26.189.195:35223)
0807 22:46:39.192974 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.192987 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.193131 (+ 144us) server_negotiation.cc:407] Connection header received
0807 22:46:39.193389 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.193395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.193653 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.193831 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:39.194117 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.195038 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.196633 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.197198 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.197317 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.197486 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.198109 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.198214 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":110,"threads_started":1}
I20260807 22:46:39.197955 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.192308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42413 (local address 127.0.0.1:50866)
0807 22:46:39.193104 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.193116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:39.193225 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:39.193925 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:39.193928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:39.193944 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:39.194044 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.194050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.195418 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.195422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.196531 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.196539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.197595 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.197598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.197687 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.197706 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 22:46:39.197765 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:46:39.197817 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":96,"threads_started":1}
I20260807 22:46:39.198101 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.192783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:45749 (local address 127.0.0.1:41120)
0807 22:46:39.193565 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.193577 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:39.193646 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:39.194328 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:39.194332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:39.194353 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:39.194448 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.194456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.195672 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.195676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.196654 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.196664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.197661 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.197666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.197757 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.197783 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:46:39.197842 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:46:39.197913 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":95,"threads_started":1}
I20260807 22:46:39.198721 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.193297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.26.189.194:45749)
0807 22:46:39.193904 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.193915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.193936 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:46:39.194006 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.194011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.194176 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.194346 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:39.194580 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.195579 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.196774 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.197521 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.198222 (+ 701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.198413 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.198457 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.198552 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":145,"threads_started":1}
I20260807 22:46:39.201218 22775 tablet_service.cc:1467] Processing CreateTablet for tablet b3249d1719044f2aa0518e5ca13e8cf6 (DEFAULT_TABLE table=test-table [id=4b2fe535fb914cebad0ffc4190070b25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:39.201422 22597 tablet_service.cc:1467] Processing CreateTablet for tablet b3249d1719044f2aa0518e5ca13e8cf6 (DEFAULT_TABLE table=test-table [id=4b2fe535fb914cebad0ffc4190070b25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:39.201426 22407 tablet_service.cc:1467] Processing CreateTablet for tablet b3249d1719044f2aa0518e5ca13e8cf6 (DEFAULT_TABLE table=test-table [id=4b2fe535fb914cebad0ffc4190070b25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:39.202145 22775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3249d1719044f2aa0518e5ca13e8cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:39.202481 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3249d1719044f2aa0518e5ca13e8cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:39.202476 22597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3249d1719044f2aa0518e5ca13e8cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:39.208706 22898 tablet_bootstrap.cc:492] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Bootstrap starting.
I20260807 22:46:39.209548 22899 tablet_bootstrap.cc:492] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Bootstrap starting.
I20260807 22:46:39.210484 22901 tablet_bootstrap.cc:492] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: Bootstrap starting.
I20260807 22:46:39.211040 22898 tablet_bootstrap.cc:654] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:39.211647 22899 tablet_bootstrap.cc:654] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:39.211942 22898 log.cc:824] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:39.212389 22901 tablet_bootstrap.cc:654] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:39.212448 22899 log.cc:824] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:39.213233 22901 log.cc:824] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:39.214526 22898 tablet_bootstrap.cc:492] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: No bootstrap required, opened a new log
I20260807 22:46:39.214712 22898 ts_tablet_manager.cc:1397] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:46:39.214910 22899 tablet_bootstrap.cc:492] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: No bootstrap required, opened a new log
I20260807 22:46:39.215097 22899 ts_tablet_manager.cc:1397] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 22:46:39.215163 22901 tablet_bootstrap.cc:492] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: No bootstrap required, opened a new log
I20260807 22:46:39.215343 22901 ts_tablet_manager.cc:1397] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:46:39.218036 22898 raft_consensus.cc:348] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.218258 22898 raft_consensus.cc:374] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:39.218338 22898 raft_consensus.cc:729] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5942ec00bb7344a9bd3f035f043d3624, State: Initialized, Role: FOLLOWER
I20260807 22:46:39.218758 22898 consensus_queue.cc:260] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.218911 22899 raft_consensus.cc:348] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.219158 22899 raft_consensus.cc:374] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:39.219218 22899 raft_consensus.cc:729] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa01af0fba946858c8fd4779ccdd86e, State: Initialized, Role: FOLLOWER
I20260807 22:46:39.219292 22901 raft_consensus.cc:348] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.219516 22901 raft_consensus.cc:374] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:39.219569 22873 heartbeater.cc:503] Master 127.26.189.254:38713 was elected leader, sending a full tablet report...
I20260807 22:46:39.219580 22901 raft_consensus.cc:729] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce6452c942945df8b1aedc4e17f85d9, State: Initialized, Role: FOLLOWER
I20260807 22:46:39.219746 22898 ts_tablet_manager.cc:1428] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:46:39.219959 22901 consensus_queue.cc:260] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.220686 22899 consensus_queue.cc:260] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.220851 22901 ts_tablet_manager.cc:1428] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:46:39.221321 22684 heartbeater.cc:503] Master 127.26.189.254:38713 was elected leader, sending a full tablet report...
I20260807 22:46:39.221545 22899 ts_tablet_manager.cc:1428] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 22:46:39.222362 22501 heartbeater.cc:503] Master 127.26.189.254:38713 was elected leader, sending a full tablet report...
W20260807 22:46:39.231505 22686 tablet.cc:2369] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:46:39.292672 22502 tablet.cc:2369] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:39.315464 22907 raft_consensus.cc:482] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:39.315652 22907 raft_consensus.cc:504] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.316684 22907 leader_election.cc:288] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5942ec00bb7344a9bd3f035f043d3624 (127.26.189.195:35223), baa01af0fba946858c8fd4779ccdd86e (127.26.189.194:45749)
I20260807 22:46:39.320430 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.317096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:46843 (local address 127.26.189.194:45749)
0807 22:46:39.317227 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.317231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.317364 (+ 133us) server_negotiation.cc:407] Connection header received
0807 22:46:39.317484 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.317486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.317528 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.317588 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:39.317869 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.318400 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.319580 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.319942 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.320044 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.320093 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.320246 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.320308 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 22:46:39.320465 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.316984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:60787 (local address 127.26.189.195:35223)
0807 22:46:39.317126 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.317131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.317291 (+ 160us) server_negotiation.cc:407] Connection header received
0807 22:46:39.317416 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.317419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.317462 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.317521 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:39.317828 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.318397 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.319582 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.319969 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.320074 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.320131 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.320281 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.320347 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:46:39.320492 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.316888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:35223 (local address 127.26.189.193:60787)
0807 22:46:39.317213 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.317225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:39.317325 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:39.317540 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:39.317543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:39.317558 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:39.317723 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.317729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.318492 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.318495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.319465 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.319475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.320084 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.320088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.320170 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.320191 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:46:39.320258 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:46:39.320326 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":93,"thread_start_us":105,"threads_started":1}
I20260807 22:46:39.320490 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.316915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:45749 (local address 127.26.189.193:46843)
0807 22:46:39.317286 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.317296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:39.317390 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:39.317604 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:39.317606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:39.317621 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:39.317765 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.317770 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.318494 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.318498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.319469 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:39.319479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.320044 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.320048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:39.320141 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.320161 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 22:46:39.320225 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:46:39.320292 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":86,"thread_start_us":91,"threads_started":1}
I20260807 22:46:39.321067 22624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3249d1719044f2aa0518e5ca13e8cf6" candidate_uuid: "0ce6452c942945df8b1aedc4e17f85d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa01af0fba946858c8fd4779ccdd86e" is_pre_election: true
I20260807 22:46:39.321110 22802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3249d1719044f2aa0518e5ca13e8cf6" candidate_uuid: "0ce6452c942945df8b1aedc4e17f85d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5942ec00bb7344a9bd3f035f043d3624" is_pre_election: true
I20260807 22:46:39.321341 22624 raft_consensus.cc:2454] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ce6452c942945df8b1aedc4e17f85d9 in term 0.
I20260807 22:46:39.321375 22802 raft_consensus.cc:2454] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ce6452c942945df8b1aedc4e17f85d9 in term 0.
I20260807 22:46:39.321794 22343 leader_election.cc:302] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ce6452c942945df8b1aedc4e17f85d9, baa01af0fba946858c8fd4779ccdd86e; no voters:
I20260807 22:46:39.322039 22907 raft_consensus.cc:2781] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:39.322122 22907 raft_consensus.cc:482] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:39.322170 22907 raft_consensus.cc:3037] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:39.323251 22907 raft_consensus.cc:504] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.323536 22907 leader_election.cc:288] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [CANDIDATE]: Term 1 election: Requested vote from peers 5942ec00bb7344a9bd3f035f043d3624 (127.26.189.195:35223), baa01af0fba946858c8fd4779ccdd86e (127.26.189.194:45749)
I20260807 22:46:39.323843 22802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3249d1719044f2aa0518e5ca13e8cf6" candidate_uuid: "0ce6452c942945df8b1aedc4e17f85d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5942ec00bb7344a9bd3f035f043d3624"
I20260807 22:46:39.323894 22624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3249d1719044f2aa0518e5ca13e8cf6" candidate_uuid: "0ce6452c942945df8b1aedc4e17f85d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa01af0fba946858c8fd4779ccdd86e"
I20260807 22:46:39.323951 22802 raft_consensus.cc:3037] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:39.323992 22624 raft_consensus.cc:3037] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:39.324936 22802 raft_consensus.cc:2454] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ce6452c942945df8b1aedc4e17f85d9 in term 1.
I20260807 22:46:39.325109 22624 raft_consensus.cc:2454] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ce6452c942945df8b1aedc4e17f85d9 in term 1.
I20260807 22:46:39.325228 22345 leader_election.cc:302] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ce6452c942945df8b1aedc4e17f85d9, 5942ec00bb7344a9bd3f035f043d3624; no voters:
I20260807 22:46:39.325394 22907 raft_consensus.cc:2781] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:39.325604 22907 raft_consensus.cc:686] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 1 LEADER]: Becoming Leader. State: Replica: 0ce6452c942945df8b1aedc4e17f85d9, State: Running, Role: LEADER
I20260807 22:46:39.325922 22907 consensus_queue.cc:237] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } }
I20260807 22:46:39.328856 22241 catalog_manager.cc:5495] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ce6452c942945df8b1aedc4e17f85d9 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "0ce6452c942945df8b1aedc4e17f85d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ce6452c942945df8b1aedc4e17f85d9" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 } health_report { overall_health: UNKNOWN } } }
I20260807 22:46:39.347334 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.343920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50878 (local address 127.26.189.193:42413)
0807 22:46:39.344041 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:39.344046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:39.344059 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:46:39.344174 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:39.344176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:39.344230 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:39.344315 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:39.344533 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.345028 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.345935 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:39.346144 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:39.346228 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:39.346561 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:39.346983 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:39.347084 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:39.347155 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 22:46:39.347206 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:46:39.352551 22802 raft_consensus.cc:1261] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 1 FOLLOWER]: Refusing update from remote peer 0ce6452c942945df8b1aedc4e17f85d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:39.352551 22624 raft_consensus.cc:1261] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 1 FOLLOWER]: Refusing update from remote peer 0ce6452c942945df8b1aedc4e17f85d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:39.353149 22913 consensus_queue.cc:1035] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5942ec00bb7344a9bd3f035f043d3624" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:39.353370 22907 consensus_queue.cc:1035] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa01af0fba946858c8fd4779ccdd86e" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 45749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:39.359983 22918 mvcc.cc:205] Tried to move back new op lower bound from 7316040906141949952 to 7316040906041597952. Current Snapshot: MvccSnapshot[applied={T|T < 7316040906141949952}]
I20260807 22:46:39.361740 22920 mvcc.cc:205] Tried to move back new op lower bound from 7316040906141949952 to 7316040906041597952. Current Snapshot: MvccSnapshot[applied={T|T < 7316040906141949952}]
I20260807 22:46:39.368395 22244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:39.368528 22244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:39.370602 22244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55b6dfaf5a0044cb8f70d8f75b9fb50f: Sending DeleteTablet for 3 replicas of tablet b3249d1719044f2aa0518e5ca13e8cf6
I20260807 22:46:39.371320 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet b3249d1719044f2aa0518e5ca13e8cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:39 UTC) from {username='slave'} at 127.0.0.1:50866
I20260807 22:46:39.371567 22597 tablet_service.cc:1514] Processing DeleteTablet for tablet b3249d1719044f2aa0518e5ca13e8cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:39 UTC) from {username='slave'} at 127.0.0.1:41120
I20260807 22:46:39.371826 22932 tablet_replica.cc:330] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9: stopping tablet replica
I20260807 22:46:39.371946 22775 tablet_service.cc:1514] Processing DeleteTablet for tablet b3249d1719044f2aa0518e5ca13e8cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:39 UTC) from {username='slave'} at 127.0.0.1:34468
I20260807 22:46:39.372195 22932 raft_consensus.cc:2229] T b3249d1719044f2aa0518e5ca13e8cf6 P 0ce6452c942945df8b1aedc4e17f85d9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:39.372279 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 22322
I20260807 22:46:39.372393 22933 tablet_replica.cc:330] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: stopping tablet replica
I20260807 22:46:39.372629 22933 raft_consensus.cc:2229] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:39.372869 22933 raft_consensus.cc:2258] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:39.373189 22934 tablet_replica.cc:330] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: stopping tablet replica
I20260807 22:46:39.373410 22934 raft_consensus.cc:2229] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:39.373615 22934 raft_consensus.cc:2258] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:39.379729 22933 ts_tablet_manager.cc:1905] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:39.380105 22934 ts_tablet_manager.cc:1905] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:39.382102 22933 ts_tablet_manager.cc:1918] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:39.382203 22933 log.cc:1196] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/b3249d1719044f2aa0518e5ca13e8cf6
I20260807 22:46:39.382460 22933 ts_tablet_manager.cc:1939] T b3249d1719044f2aa0518e5ca13e8cf6 P baa01af0fba946858c8fd4779ccdd86e: Deleting consensus metadata
I20260807 22:46:39.383589 22934 ts_tablet_manager.cc:1918] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:39.383757 22934 log.cc:1196] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/b3249d1719044f2aa0518e5ca13e8cf6
I20260807 22:46:39.383852 22222 catalog_manager.cc:4841] TS baa01af0fba946858c8fd4779ccdd86e (127.26.189.194:45749): tablet b3249d1719044f2aa0518e5ca13e8cf6 (table test-table [id=4b2fe535fb914cebad0ffc4190070b25]) successfully deleted
I20260807 22:46:39.384140 22934 ts_tablet_manager.cc:1939] T b3249d1719044f2aa0518e5ca13e8cf6 P 5942ec00bb7344a9bd3f035f043d3624: Deleting consensus metadata
I20260807 22:46:39.385497 22224 catalog_manager.cc:4841] TS 5942ec00bb7344a9bd3f035f043d3624 (127.26.189.195:35223): tablet b3249d1719044f2aa0518e5ca13e8cf6 (table test-table [id=4b2fe535fb914cebad0ffc4190070b25]) successfully deleted
W20260807 22:46:39.392593 22222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:42413 (error 108)
I20260807 22:46:39.393421 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 22504
I20260807 22:46:39.393635 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:39.393335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42413 (local address 127.0.0.1:50890)
0807 22:46:39.393473 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:39.393550 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 22:46:39.398192 22222 catalog_manager.cc:4584] TS 0ce6452c942945df8b1aedc4e17f85d9 (127.26.189.193:42413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3249d1719044f2aa0518e5ca13e8cf6: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42413: connect: Connection refused (error 111)
I20260807 22:46:39.405020 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 22688
I20260807 22:46:39.422259 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 22197
2026-08-07T22:46:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1093 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:39 dist-test-slave-4tdj krb5kdc[22939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:39 dist-test-slave-4tdj krb5kdc[22939](info): set up 2 sockets
Aug 07 22:46:39 dist-test-slave-4tdj krb5kdc[22939](info): commencing operation
krb5kdc: starting...
W20260807 22:46:40.552340 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:40 dist-test-slave-4tdj krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:46:40.669401 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38635
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:40.776937 22957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:40.777179 22957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:40.777216 22957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:40.781086 22957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:40.781136 22957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:40.781153 22957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:40.781167 22957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:40.781181 22957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:40.786175 22957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:40.787133 22957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:40.788084 22957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:40.796033 22968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:40.796094 22966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:40.796087 22971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:40.796283 22957 server_base.cc:1034] running on GCE node
I20260807 22:46:40.796794 22957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:40.797780 22957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:40.798926 22957 hybrid_clock.cc:648] HybridClock initialized: now 1786142800798914 us; error 27 us; skew 500 ppm
Aug 07 22:46:40 dist-test-slave-4tdj krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142800, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:40.801576 22957 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:46:40.802418 22957 webserver.cc:460] Webserver started at http://127.26.189.254:37351/ using document root <none> and password file <none>
I20260807 22:46:40.802917 22957 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:40.802973 22957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:40.803169 22957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:40.805073 22957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "072d2f711da04da98c0d06796802a9e7"
format_stamp: "Formatted at 2026-08-07 22:46:40 on dist-test-slave-4tdj"
server_key: "fde9702e704dd7299689c6a3502e437e"
server_key_iv: "8329231c0b5aba31b7bf2f59c2eef965"
server_key_version: "encryptionkey@0"
I20260807 22:46:40.805609 22957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "072d2f711da04da98c0d06796802a9e7"
format_stamp: "Formatted at 2026-08-07 22:46:40 on dist-test-slave-4tdj"
server_key: "fde9702e704dd7299689c6a3502e437e"
server_key_iv: "8329231c0b5aba31b7bf2f59c2eef965"
server_key_version: "encryptionkey@0"
I20260807 22:46:40.809139 22957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:46:40.811357 22979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:40.812559 22957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:46:40.812666 22957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "072d2f711da04da98c0d06796802a9e7"
format_stamp: "Formatted at 2026-08-07 22:46:40 on dist-test-slave-4tdj"
server_key: "fde9702e704dd7299689c6a3502e437e"
server_key_iv: "8329231c0b5aba31b7bf2f59c2eef965"
server_key_version: "encryptionkey@0"
I20260807 22:46:40.812753 22957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:40.830710 22957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:40.833299 22957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:40.833442 22957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:40.839936 22957 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38635
I20260807 22:46:40.839978 23054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38635 every 8 connection(s)
I20260807 22:46:40.840822 22957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:40.843473 23055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:40.844964 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 22957
I20260807 22:46:40.845120 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:40.845420 27383 external_mini_cluster.cc:1428] Setting key d7c35a045a67fd03bca3ec897a046954
I20260807 22:46:40.849889 23055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7: Bootstrap starting.
I20260807 22:46:40.852398 23055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:40.853259 23055 log.cc:824] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7: Log is configured to *not* fsync() on all Append() calls
Aug 07 22:46:40 dist-test-slave-4tdj krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:40.855226 23055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7: No bootstrap required, opened a new log
I20260807 22:46:40.859485 23055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072d2f711da04da98c0d06796802a9e7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38635 } }
I20260807 22:46:40.859748 23055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:40.859819 23055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072d2f711da04da98c0d06796802a9e7, State: Initialized, Role: FOLLOWER
I20260807 22:46:40.860270 23055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072d2f711da04da98c0d06796802a9e7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38635 } }
I20260807 22:46:40.860356 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:40.846810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.26.189.254:38635)
0807 22:46:40.847222 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:40.847241 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:40.847282 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:46:40.848076 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:40.848100 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:40.848494 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:40.848859 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:40.849521 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:40.850976 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:40.851869 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:40.852532 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:40.852693 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:40.855355 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:40.855361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:40.855376 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:40.855405 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:40.857266 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:40.857710 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:40.857716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:40.857721 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:40.857853 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:40.858146 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:40.858149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:40.858150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:40.858655 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:40.858850 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:40.859124 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 22:46:40.859402 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":112,"threads_started":1}
I20260807 22:46:40.860426 23055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:40.860481 23055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:40.860584 23055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:40.861418 23055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072d2f711da04da98c0d06796802a9e7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38635 } }
I20260807 22:46:40.861771 23055 leader_election.cc:302] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 072d2f711da04da98c0d06796802a9e7; no voters:
I20260807 22:46:40.862084 23055 leader_election.cc:288] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:40.862221 23060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:40.862454 23060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [term 1 LEADER]: Becoming Leader. State: Replica: 072d2f711da04da98c0d06796802a9e7, State: Running, Role: LEADER
I20260807 22:46:40.862746 23060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072d2f711da04da98c0d06796802a9e7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38635 } }
I20260807 22:46:40.863240 23055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:40.863781 23060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 072d2f711da04da98c0d06796802a9e7. Latest consensus state: current_term: 1 leader_uuid: "072d2f711da04da98c0d06796802a9e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072d2f711da04da98c0d06796802a9e7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38635 } } }
I20260807 22:46:40.863893 23060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:40.864296 23063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "072d2f711da04da98c0d06796802a9e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "072d2f711da04da98c0d06796802a9e7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38635 } } }
I20260807 22:46:40.864399 23063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:40.864773 23068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:40.867487 23068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:40.872387 23068 catalog_manager.cc:1347] Generated new cluster ID: 0f1246eea1de478da7c66256cb3a6412
I20260807 22:46:40.872449 23068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:40.886917 23068 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:40.888139 23068 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:40.896932 23068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7: Generated new TSK 0
I20260807 22:46:40.897621 23068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:46:40.925333 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38635
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:41.038091 23089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:41.038386 23089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:41.038443 23089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:41.042393 23089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:41.042448 23089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:41.042536 23089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:41.047705 23089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:41.048707 23089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:41.049682 23089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:41.059242 23099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.059266 23098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.059238 23101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:41.059291 23089 server_base.cc:1034] running on GCE node
I20260807 22:46:41.059872 23089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:41.060453 23089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:41.061659 23089 hybrid_clock.cc:648] HybridClock initialized: now 1786142801061634 us; error 38 us; skew 500 ppm
Aug 07 22:46:41 dist-test-slave-4tdj krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142801, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:41.064524 23089 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:46:41.065377 23089 webserver.cc:460] Webserver started at http://127.26.189.193:35267/ using document root <none> and password file <none>
I20260807 22:46:41.065934 23089 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:41.065995 23089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:41.066157 23089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:41.068166 23089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "487574de74ad499099d5957de49eb869"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "1803ab92f698f62732cd99f5f1c5c434"
server_key_iv: "f91d86af26c0b6119ab98287731e4f5a"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.068674 23089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "487574de74ad499099d5957de49eb869"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "1803ab92f698f62732cd99f5f1c5c434"
server_key_iv: "f91d86af26c0b6119ab98287731e4f5a"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.072237 23089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:46:41.074584 23110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.075743 23089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:46:41.075848 23089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "487574de74ad499099d5957de49eb869"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "1803ab92f698f62732cd99f5f1c5c434"
server_key_iv: "f91d86af26c0b6119ab98287731e4f5a"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.075938 23089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:41.090150 23089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:41.092912 23089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:41.093084 23089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:41.093654 23089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:41.094482 23089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:41.094538 23089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.094594 23089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:41.094619 23089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.103289 23089 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:42697
I20260807 22:46:41.103320 23271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:42697 every 8 connection(s)
I20260807 22:46:41.104126 23089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:41.112214 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 23089
I20260807 22:46:41.112368 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:41.112730 27383 external_mini_cluster.cc:1428] Setting key 322981b8dcb2dc0d18e7b3dfdbefee1e
Aug 07 22:46:41 dist-test-slave-4tdj krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142801, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:41.118182 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.105892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:40671 (local address 127.26.189.254:38635)
0807 22:46:41.106023 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.106026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.106833 (+ 807us) server_negotiation.cc:407] Connection header received
0807 22:46:41.108151 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.108154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.108214 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.108289 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:41.109450 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.109989 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.111159 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.111390 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.111481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.114282 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:41.114285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:41.114309 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:41.114336 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:41.115893 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.116536 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.116539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.116540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.116636 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.117018 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.117022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.117025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.117222 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:41.117323 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.117849 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.118000 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:46:41.118999 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.106222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38635 (local address 127.26.189.193:40671)
0807 22:46:41.106690 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.106733 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.107930 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.108468 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.108475 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.108819 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:41.109297 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.109312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.110113 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.110116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.111038 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.111049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.111538 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.111542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.111714 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.112294 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:41.112315 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:41.114117 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:41.116038 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:41.116045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:41.116061 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:41.116392 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:41.116760 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:41.116762 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:41.116764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:41.116910 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:41.117354 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:41.117361 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:41.117608 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.117787 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.118116 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":122,"threads_started":1}
I20260807 22:46:41.120318 23272 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38635
I20260807 22:46:41.120575 23272 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:41.121148 23272 heartbeater.cc:511] Master 127.26.189.254:38635 requested a full tablet report, sending...
I20260807 22:46:41.122845 23003 ts_manager.cc:194] Registered new tserver with Master: 487574de74ad499099d5957de49eb869 (127.26.189.193:42697)
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:46:41.124145 23003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:40671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:46:41.141378 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38635
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:41.252630 23281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:41.252895 23281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:41.252940 23281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:41.257043 23281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:41.257102 23281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:41.257161 23281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:41.262358 23281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:41.263342 23281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:41.264364 23281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:41.273869 23287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.273878 23290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.273878 23286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:41.274262 23281 server_base.cc:1034] running on GCE node
I20260807 22:46:41.274688 23281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:41.275314 23281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:41.276463 23281 hybrid_clock.cc:648] HybridClock initialized: now 1786142801276455 us; error 27 us; skew 500 ppm
Aug 07 22:46:41 dist-test-slave-4tdj krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142801, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:41.279248 23281 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:46:41.280232 23281 webserver.cc:460] Webserver started at http://127.26.189.194:35729/ using document root <none> and password file <none>
I20260807 22:46:41.280781 23281 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:41.280853 23281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:41.281066 23281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:41.283170 23281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "e8acd3472a2ee093eef31c268bdde422"
server_key_iv: "bb7cc66189779f5c4af2cf55629fdc1a"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.283743 23281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "e8acd3472a2ee093eef31c268bdde422"
server_key_iv: "bb7cc66189779f5c4af2cf55629fdc1a"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.287611 23281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:41.289934 23305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.291217 23281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:41.291333 23281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "e8acd3472a2ee093eef31c268bdde422"
server_key_iv: "bb7cc66189779f5c4af2cf55629fdc1a"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.291432 23281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:41.313762 23281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:41.316747 23281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:41.316934 23281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:41.317577 23281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:41.318483 23281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:41.318539 23281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.318598 23281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:41.318629 23281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.327865 23281 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:40995
I20260807 22:46:41.327886 23477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:40995 every 8 connection(s)
I20260807 22:46:41.328760 23281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:41.337684 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 23281
I20260807 22:46:41.337815 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:41.338094 27383 external_mini_cluster.cc:1428] Setting key c286f96d0004cab9c4d9360ca1f7ce08
Aug 07 22:46:41 dist-test-slave-4tdj krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142801, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:41.342284 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.330582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:54745 (local address 127.26.189.254:38635)
0807 22:46:41.330713 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.330717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.331631 (+ 914us) server_negotiation.cc:407] Connection header received
0807 22:46:41.332933 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.332936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.332993 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.333081 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:41.334213 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.334790 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.336020 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.336257 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.336347 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.338858 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:41.338861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:41.338863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:41.338888 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:41.340302 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.340816 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.340819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.340821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.340911 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.341192 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.341195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.341196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.341395 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:41.341502 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.342071 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.342170 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:46:41.343335 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.330989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38635 (local address 127.26.189.194:54745)
0807 22:46:41.331467 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.331509 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.332722 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.333261 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.333268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.333605 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:41.334064 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.334078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.334914 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.334917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.335899 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.335910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.336399 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.336404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.336568 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.337071 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:41.337086 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:41.338693 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:41.340418 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:41.340422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:41.340433 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:41.340723 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:41.340989 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:41.340991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:41.340992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:41.341117 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:41.341480 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:41.341485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:41.341827 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.342021 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.342361 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":130,"threads_started":1}
I20260807 22:46:41.344449 23478 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38635
I20260807 22:46:41.344707 23478 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:41.345266 23478 heartbeater.cc:511] Master 127.26.189.254:38635 requested a full tablet report, sending...
I20260807 22:46:41.346328 23003 ts_manager.cc:194] Registered new tserver with Master: 7c0153e3b24f48b9ad13e2c45d4a1ac7 (127.26.189.194:40995)
I20260807 22:46:41.346910 23003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:54745
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:46:41.365424 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38635
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:41.473455 23484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:41.473682 23484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:41.473722 23484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:41.477645 23484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:41.477702 23484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:41.477769 23484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:41.483096 23484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:38635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:41.484076 23484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:41.485198 23484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:41.494982 23491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.495019 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:41.495229 23484 server_base.cc:1034] running on GCE node
W20260807 22:46:41.495286 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:41.495657 23484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:41.496310 23484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:41.497478 23484 hybrid_clock.cc:648] HybridClock initialized: now 1786142801497448 us; error 44 us; skew 500 ppm
Aug 07 22:46:41 dist-test-slave-4tdj krb5kdc[22939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142801, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:41.500224 23484 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:46:41.501075 23484 webserver.cc:460] Webserver started at http://127.26.189.195:34407/ using document root <none> and password file <none>
I20260807 22:46:41.501602 23484 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:41.501672 23484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:41.501868 23484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:41.503861 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "c12be7a638c14ef2896fd2c4909c9ae7"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "d7c2e46f3ba901895234324cc28efb44"
server_key_iv: "6da48d1a36a03c32be19681f9f835953"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.504390 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "c12be7a638c14ef2896fd2c4909c9ae7"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "d7c2e46f3ba901895234324cc28efb44"
server_key_iv: "6da48d1a36a03c32be19681f9f835953"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.508008 23484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:46:41.510239 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.511430 23484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:46:41.511533 23484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "c12be7a638c14ef2896fd2c4909c9ae7"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "d7c2e46f3ba901895234324cc28efb44"
server_key_iv: "6da48d1a36a03c32be19681f9f835953"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.511626 23484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:41.527791 23484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:41.530759 23484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:41.530930 23484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:41.531487 23484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:41.532276 23484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:41.532322 23484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.532373 23484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:41.532402 23484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.540627 23484 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:44573
I20260807 22:46:41.540663 23679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:44573 every 8 connection(s)
I20260807 22:46:41.541455 23484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:46:41 dist-test-slave-4tdj krb5kdc[22939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142801, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:41.551306 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 23484
I20260807 22:46:41.551436 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:41.551728 27383 external_mini_cluster.cc:1428] Setting key fde8ce4511832ba3781e1866e8a4d16e
I20260807 22:46:41.553804 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.543212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:57949 (local address 127.26.189.254:38635)
0807 22:46:41.543358 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.543362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.544108 (+ 746us) server_negotiation.cc:407] Connection header received
0807 22:46:41.545350 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.545354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.545411 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.545484 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:41.546636 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.547174 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.548261 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.548474 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.548570 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.550743 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:41.550746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:41.550748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:41.550770 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:41.551988 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.552433 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.552436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.552437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.552511 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.552816 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.552819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.552821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.552984 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:41.553068 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.553536 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.553652 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:46:41.554551 23682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.543500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:38635 (local address 127.26.189.195:57949)
0807 22:46:41.543942 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.543983 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.545135 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.545664 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.545671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.545989 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:41.546498 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.546513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.547289 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.547292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.548145 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.548155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.548609 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.548613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.548767 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.549228 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:41.549243 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:41.550590 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:41.552091 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:41.552096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:41.552109 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:41.552338 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:41.552600 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:41.552603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:41.552605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:41.552730 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:41.553089 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:41.553095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:41.553307 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.553469 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.553758 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
I20260807 22:46:41.555467 23680 heartbeater.cc:347] Connected to a master server at 127.26.189.254:38635
I20260807 22:46:41.555661 23680 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:41.556072 23680 heartbeater.cc:511] Master 127.26.189.254:38635 requested a full tablet report, sending...
I20260807 22:46:41.556927 23003 ts_manager.cc:194] Registered new tserver with Master: c12be7a638c14ef2896fd2c4909c9ae7 (127.26.189.195:44573)
I20260807 22:46:41.557454 23003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:57949
I20260807 22:46:41.565133 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:41.574249 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.567136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.26.189.254:38635)
0807 22:46:41.567258 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.567262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.567432 (+ 170us) server_negotiation.cc:407] Connection header received
0807 22:46:41.567568 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.567571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.567613 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.567674 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:41.568380 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.568876 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.569671 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.569886 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.569967 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.571410 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:41.571414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:41.571416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:41.571440 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:41.572584 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.573038 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.573042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.573044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.573116 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:41.573416 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:41.573419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:41.573421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:41.573613 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:41.573742 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.573994 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.574104 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:46:41.576928 23003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:41.580080 23003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:41.593132 23696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.587737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:44573 (local address 127.0.0.1:53168)
0807 22:46:41.588033 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.588076 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.588190 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.588722 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.588729 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.588757 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:41.588896 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.588903 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.590416 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.590419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.591694 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.591703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.592720 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.592723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.592829 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.592874 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.592943 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.593024 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260807 22:46:41.593840 23699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.588411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:40995 (local address 127.0.0.1:45306)
0807 22:46:41.588642 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.588655 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.588732 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.589640 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.589643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.589662 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:41.589768 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.589774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.591029 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.591032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.592148 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.592159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.593443 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.593447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.593537 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.593559 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.593639 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.593708 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":39,"thread_start_us":98,"threads_started":1}
I20260807 22:46:41.593920 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.588609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.26.189.194:40995)
0807 22:46:41.589202 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.589213 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.589234 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:46:41.589304 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.589309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.589475 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.589906 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:41.590060 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.590897 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.592288 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.593120 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.593237 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.593355 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.593694 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.593789 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":153,"threads_started":1}
I20260807 22:46:41.594147 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.587888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.26.189.195:44573)
0807 22:46:41.588198 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.588207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.588229 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:46:41.588334 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.588338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.588535 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.589076 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:41.589218 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.590273 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.591804 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.592588 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.593664 (+ 1076us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.593786 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.593830 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.593905 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
I20260807 22:46:41.594925 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.588727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42697 (local address 127.0.0.1:36148)
0807 22:46:41.588927 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.588939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.589019 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.589832 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.589835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.589854 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:41.589948 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.589954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.591548 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.591551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.592704 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.592715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.593796 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.593800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.593886 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.593909 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.593959 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.594807 (+ 848us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":39,"thread_start_us":88,"threads_started":1}
I20260807 22:46:41.595456 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.589145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.26.189.193:42697)
0807 22:46:41.589385 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.589401 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.589422 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:46:41.589488 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.589493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.589685 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.590116 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:41.590232 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.591396 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.592844 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.593673 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.595112 (+ 1439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.595213 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.595255 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.595322 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260807 22:46:41.596880 23367 tablet_service.cc:1467] Processing CreateTablet for tablet bc9d35340d8e4d6a811c7d75cd99ffdf (DEFAULT_TABLE table=test-table [id=d60b6445584d453ea3e81c03b10e1789]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:41.596872 23576 tablet_service.cc:1467] Processing CreateTablet for tablet bc9d35340d8e4d6a811c7d75cd99ffdf (DEFAULT_TABLE table=test-table [id=d60b6445584d453ea3e81c03b10e1789]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:41.597962 23367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9d35340d8e4d6a811c7d75cd99ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:41.597962 23576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9d35340d8e4d6a811c7d75cd99ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:41.600781 23179 tablet_service.cc:1467] Processing CreateTablet for tablet bc9d35340d8e4d6a811c7d75cd99ffdf (DEFAULT_TABLE table=test-table [id=d60b6445584d453ea3e81c03b10e1789]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:41.601616 23179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9d35340d8e4d6a811c7d75cd99ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:41.605244 23703 tablet_bootstrap.cc:492] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Bootstrap starting.
I20260807 22:46:41.605883 23704 tablet_bootstrap.cc:492] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Bootstrap starting.
I20260807 22:46:41.608052 23703 tablet_bootstrap.cc:654] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:41.608126 23705 tablet_bootstrap.cc:492] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869: Bootstrap starting.
I20260807 22:46:41.608150 23704 tablet_bootstrap.cc:654] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:41.609366 23704 log.cc:824] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:41.609378 23703 log.cc:824] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:41.610160 23705 tablet_bootstrap.cc:654] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:41.611256 23704 tablet_bootstrap.cc:492] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: No bootstrap required, opened a new log
I20260807 22:46:41.611357 23705 log.cc:824] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:41.611400 23703 tablet_bootstrap.cc:492] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: No bootstrap required, opened a new log
I20260807 22:46:41.611438 23704 ts_tablet_manager.cc:1397] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:46:41.611586 23703 ts_tablet_manager.cc:1397] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 22:46:41.613232 23705 tablet_bootstrap.cc:492] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869: No bootstrap required, opened a new log
I20260807 22:46:41.613395 23705 ts_tablet_manager.cc:1397] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:46:41.614331 23704 raft_consensus.cc:348] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.614568 23704 raft_consensus.cc:374] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:41.614604 23704 raft_consensus.cc:729] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0153e3b24f48b9ad13e2c45d4a1ac7, State: Initialized, Role: FOLLOWER
I20260807 22:46:41.614976 23704 consensus_queue.cc:260] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.615196 23703 raft_consensus.cc:348] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.615422 23703 raft_consensus.cc:374] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:41.615466 23703 raft_consensus.cc:729] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12be7a638c14ef2896fd2c4909c9ae7, State: Initialized, Role: FOLLOWER
I20260807 22:46:41.615619 23478 heartbeater.cc:503] Master 127.26.189.254:38635 was elected leader, sending a full tablet report...
I20260807 22:46:41.615939 23703 consensus_queue.cc:260] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.616318 23704 ts_tablet_manager.cc:1428] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:46:41.616895 23703 ts_tablet_manager.cc:1428] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:46:41.616971 23680 heartbeater.cc:503] Master 127.26.189.254:38635 was elected leader, sending a full tablet report...
I20260807 22:46:41.617197 23705 raft_consensus.cc:348] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.617419 23705 raft_consensus.cc:374] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:41.617483 23705 raft_consensus.cc:729] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487574de74ad499099d5957de49eb869, State: Initialized, Role: FOLLOWER
I20260807 22:46:41.617863 23705 consensus_queue.cc:260] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.618537 23272 heartbeater.cc:503] Master 127.26.189.254:38635 was elected leader, sending a full tablet report...
I20260807 22:46:41.618780 23705 ts_tablet_manager.cc:1428] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:46:41.654721 23712 raft_consensus.cc:482] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:41.654902 23712 raft_consensus.cc:504] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.655937 23712 leader_election.cc:288] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12be7a638c14ef2896fd2c4909c9ae7 (127.26.189.195:44573), 487574de74ad499099d5957de49eb869 (127.26.189.193:42697)
I20260807 22:46:41.660773 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.656205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:44573 (local address 127.26.189.194:53963)
0807 22:46:41.656331 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.656346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.656437 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.656781 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.656785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.656806 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:41.656932 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.656939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.657888 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.657891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.659170 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.659181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.660091 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.660096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.660356 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.660402 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.660497 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.660584 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":71}
I20260807 22:46:41.660800 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.656632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:50045 (local address 127.26.189.193:42697)
0807 22:46:41.656751 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.656755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.656919 (+ 164us) server_negotiation.cc:407] Connection header received
0807 22:46:41.657028 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.657031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.657079 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.657150 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:41.657385 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.658002 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.659381 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.659952 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.660063 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.660117 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.660583 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.660680 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:46:41.660800 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.656324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:53963 (local address 127.26.189.195:44573)
0807 22:46:41.656475 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.656480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.656495 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:46:41.656548 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.656551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.656599 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.656674 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:41.657035 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.657765 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.659300 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.659939 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.660069 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.660130 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.660598 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.660680 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 22:46:41.660784 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.656481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42697 (local address 127.26.189.194:50045)
0807 22:46:41.656845 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.656858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:41.656949 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:41.657171 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:41.657173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:41.657187 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:41.657292 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.657296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.658105 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.658108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.659272 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:41.659280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.660091 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.660096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:41.660361 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.660408 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:46:41.660534 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 22:46:41.660587 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":80,"thread_start_us":136,"threads_started":1}
I20260807 22:46:41.661689 23210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9d35340d8e4d6a811c7d75cd99ffdf" candidate_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487574de74ad499099d5957de49eb869" is_pre_election: true
I20260807 22:46:41.661733 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9d35340d8e4d6a811c7d75cd99ffdf" candidate_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" is_pre_election: true
I20260807 22:46:41.661958 23210 raft_consensus.cc:2454] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0153e3b24f48b9ad13e2c45d4a1ac7 in term 0.
I20260807 22:46:41.662004 23611 raft_consensus.cc:2454] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0153e3b24f48b9ad13e2c45d4a1ac7 in term 0.
I20260807 22:46:41.662475 23309 leader_election.cc:302] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487574de74ad499099d5957de49eb869, 7c0153e3b24f48b9ad13e2c45d4a1ac7; no voters:
I20260807 22:46:41.662731 23712 raft_consensus.cc:2781] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:41.662830 23712 raft_consensus.cc:482] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:41.662884 23712 raft_consensus.cc:3037] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:41.663949 23712 raft_consensus.cc:504] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.664265 23712 leader_election.cc:288] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [CANDIDATE]: Term 1 election: Requested vote from peers c12be7a638c14ef2896fd2c4909c9ae7 (127.26.189.195:44573), 487574de74ad499099d5957de49eb869 (127.26.189.193:42697)
I20260807 22:46:41.664608 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9d35340d8e4d6a811c7d75cd99ffdf" candidate_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12be7a638c14ef2896fd2c4909c9ae7"
I20260807 22:46:41.664721 23210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc9d35340d8e4d6a811c7d75cd99ffdf" candidate_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "487574de74ad499099d5957de49eb869"
I20260807 22:46:41.664752 23611 raft_consensus.cc:3037] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:41.664831 23210 raft_consensus.cc:3037] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:41.666054 23210 raft_consensus.cc:2454] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0153e3b24f48b9ad13e2c45d4a1ac7 in term 1.
I20260807 22:46:41.666087 23611 raft_consensus.cc:2454] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0153e3b24f48b9ad13e2c45d4a1ac7 in term 1.
I20260807 22:46:41.666386 23309 leader_election.cc:302] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 487574de74ad499099d5957de49eb869, 7c0153e3b24f48b9ad13e2c45d4a1ac7; no voters:
I20260807 22:46:41.666687 23712 raft_consensus.cc:2781] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:41.666895 23712 raft_consensus.cc:686] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 7c0153e3b24f48b9ad13e2c45d4a1ac7, State: Running, Role: LEADER
I20260807 22:46:41.667212 23712 consensus_queue.cc:237] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } }
I20260807 22:46:41.670421 23003 catalog_manager.cc:5495] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0153e3b24f48b9ad13e2c45d4a1ac7 (127.26.189.194). New cstate: current_term: 1 leader_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0153e3b24f48b9ad13e2c45d4a1ac7" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 40995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 } health_report { overall_health: UNKNOWN } } }
I20260807 22:46:41.696967 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.693500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.26.189.194:40995)
0807 22:46:41.693620 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:41.693624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:41.693637 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:46:41.693813 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:41.693816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:41.693872 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:41.693966 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:41.694179 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.694724 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.695563 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:41.695777 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:41.695854 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:41.696170 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:41.696556 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:41.696665 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:41.696772 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 22:46:41.696810 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:46:41.702517 23210 raft_consensus.cc:1261] T bc9d35340d8e4d6a811c7d75cd99ffdf P 487574de74ad499099d5957de49eb869 [term 1 FOLLOWER]: Refusing update from remote peer 7c0153e3b24f48b9ad13e2c45d4a1ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:41.702512 23611 raft_consensus.cc:1261] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 1 FOLLOWER]: Refusing update from remote peer 7c0153e3b24f48b9ad13e2c45d4a1ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:41.703217 23716 consensus_queue.cc:1035] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "487574de74ad499099d5957de49eb869" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 42697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:41.703400 23712 consensus_queue.cc:1035] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12be7a638c14ef2896fd2c4909c9ae7" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:41.712116 23723 mvcc.cc:205] Tried to move back new op lower bound from 7316040915766669312 to 7316040915631345664. Current Snapshot: MvccSnapshot[applied={T|T < 7316040915766669312}]
I20260807 22:46:41.718900 23002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:41.719059 23002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:41.721570 23002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 072d2f711da04da98c0d06796802a9e7: Sending DeleteTablet for 3 replicas of tablet bc9d35340d8e4d6a811c7d75cd99ffdf
I20260807 22:46:41.722318 23576 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9d35340d8e4d6a811c7d75cd99ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:41 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:53168
I20260807 22:46:41.722600 23179 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9d35340d8e4d6a811c7d75cd99ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:41 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:36148
I20260807 22:46:41.722599 23367 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9d35340d8e4d6a811c7d75cd99ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:41 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:45306
I20260807 22:46:41.723075 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 23089
I20260807 22:46:41.723100 23737 tablet_replica.cc:330] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: stopping tablet replica
I20260807 22:46:41.723218 23733 tablet_replica.cc:330] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: stopping tablet replica
I20260807 22:46:41.723541 23737 raft_consensus.cc:2229] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:41.723547 23733 raft_consensus.cc:2229] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:41.723861 23733 raft_consensus.cc:2258] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:41.724135 23737 raft_consensus.cc:2258] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:41.728545 23737 ts_tablet_manager.cc:1905] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:41.732002 23737 ts_tablet_manager.cc:1918] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:41.732149 23737 log.cc:1196] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/bc9d35340d8e4d6a811c7d75cd99ffdf
I20260807 22:46:41.732573 23737 ts_tablet_manager.cc:1939] T bc9d35340d8e4d6a811c7d75cd99ffdf P 7c0153e3b24f48b9ad13e2c45d4a1ac7: Deleting consensus metadata
I20260807 22:46:41.734078 22983 catalog_manager.cc:4841] TS 7c0153e3b24f48b9ad13e2c45d4a1ac7 (127.26.189.194:40995): tablet bc9d35340d8e4d6a811c7d75cd99ffdf (table test-table [id=d60b6445584d453ea3e81c03b10e1789]) successfully deleted
I20260807 22:46:41.740320 23733 ts_tablet_manager.cc:1905] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:41.742558 23733 ts_tablet_manager.cc:1918] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:41.742656 23733 log.cc:1196] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/bc9d35340d8e4d6a811c7d75cd99ffdf
I20260807 22:46:41.742954 23733 ts_tablet_manager.cc:1939] T bc9d35340d8e4d6a811c7d75cd99ffdf P c12be7a638c14ef2896fd2c4909c9ae7: Deleting consensus metadata
I20260807 22:46:41.744009 22983 catalog_manager.cc:4841] TS c12be7a638c14ef2896fd2c4909c9ae7 (127.26.189.195:44573): tablet bc9d35340d8e4d6a811c7d75cd99ffdf (table test-table [id=d60b6445584d453ea3e81c03b10e1789]) successfully deleted
W20260807 22:46:41.746824 22983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:42697 (error 108)
I20260807 22:46:41.747704 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 23281
I20260807 22:46:41.747895 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:41.747593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:42697 (local address 127.0.0.1:36154)
0807 22:46:41.747738 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:41.747807 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:46:41.752717 22983 catalog_manager.cc:4584] TS 487574de74ad499099d5957de49eb869 (127.26.189.193:42697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc9d35340d8e4d6a811c7d75cd99ffdf: Network error: Client connection negotiation failed: client connection to 127.26.189.193:42697: connect: Connection refused (error 111)
I20260807 22:46:41.761030 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 23484
I20260807 22:46:41.771260 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 22957
2026-08-07T22:46:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2348 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T22:46:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:41Z Disabled control of system clock
I20260807 22:46:41.803596 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:37091
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:37091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:46:41.911384 23744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:41.911648 23744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:41.911680 23744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:41.915594 23744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:41.915647 23744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:41.915660 23744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:41.915675 23744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:41.915688 23744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:41.920413 23744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:37091
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:37091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:41.921525 23744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:41.922886 23744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:41.930958 23755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.930975 23751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:41.930948 23750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:41.931318 23744 server_base.cc:1034] running on GCE node
I20260807 22:46:41.931850 23744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:41.932873 23744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:41.934036 23744 hybrid_clock.cc:648] HybridClock initialized: now 1786142801934014 us; error 32 us; skew 500 ppm
I20260807 22:46:41.935806 23744 webserver.cc:460] Webserver started at http://127.26.189.254:39203/ using document root <none> and password file <none>
I20260807 22:46:41.936338 23744 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:41.936404 23744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:41.936590 23744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:41.938575 23744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "1722e4563eef485cac877a2c3414c356"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "0a63a9ff7d3e86058c32c2e8ad324532"
server_key_iv: "c86bb23ae6ef495df24cffbd2583fe52"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.939112 23744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "1722e4563eef485cac877a2c3414c356"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "0a63a9ff7d3e86058c32c2e8ad324532"
server_key_iv: "c86bb23ae6ef495df24cffbd2583fe52"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.942657 23744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 22:46:41.945040 23763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:41.946283 23744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:41.946450 23744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "1722e4563eef485cac877a2c3414c356"
format_stamp: "Formatted at 2026-08-07 22:46:41 on dist-test-slave-4tdj"
server_key: "0a63a9ff7d3e86058c32c2e8ad324532"
server_key_iv: "c86bb23ae6ef495df24cffbd2583fe52"
server_key_version: "encryptionkey@0"
I20260807 22:46:41.946555 23744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:41.985343 23744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:41.986039 23744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:41.986214 23744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:41.992928 23744 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:37091
I20260807 22:46:41.992940 23834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:37091 every 8 connection(s)
I20260807 22:46:41.993810 23744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:41.996840 23836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:41.999498 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 23744
I20260807 22:46:41.999630 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:41.999881 27383 external_mini_cluster.cc:1428] Setting key 204983d55714ac2fa618e8c287186f18
I20260807 22:46:42.002550 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: Bootstrap starting.
I20260807 22:46:42.005136 23836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:42.005923 23836 log.cc:824] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:42.007656 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: No bootstrap required, opened a new log
I20260807 22:46:42.009379 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.001234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50694 (local address 127.26.189.254:37091)
0807 22:46:42.001897 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.001919 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.001959 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:46:42.002779 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.002804 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.003278 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.003627 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:42.004054 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.005471 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.006411 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.006895 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.007039 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.007821 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:42.007826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:42.007838 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:42.008201 (+ 363us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:42.008253 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:42.008271 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:42.008355 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.008497 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.008736 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
I20260807 22:46:42.011247 23836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1722e4563eef485cac877a2c3414c356" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 37091 } }
I20260807 22:46:42.011502 23836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:42.011559 23836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1722e4563eef485cac877a2c3414c356, State: Initialized, Role: FOLLOWER
I20260807 22:46:42.011957 23836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1722e4563eef485cac877a2c3414c356" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 37091 } }
I20260807 22:46:42.012080 23836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:42.012141 23836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:42.012228 23836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:42.013077 23836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1722e4563eef485cac877a2c3414c356" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 37091 } }
I20260807 22:46:42.013383 23836 leader_election.cc:302] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1722e4563eef485cac877a2c3414c356; no voters:
I20260807 22:46:42.013649 23836 leader_election.cc:288] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:42.013741 23846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:42.013959 23846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [term 1 LEADER]: Becoming Leader. State: Replica: 1722e4563eef485cac877a2c3414c356, State: Running, Role: LEADER
I20260807 22:46:42.014251 23846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1722e4563eef485cac877a2c3414c356" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 37091 } }
I20260807 22:46:42.014864 23836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:42.015798 23847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1722e4563eef485cac877a2c3414c356" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1722e4563eef485cac877a2c3414c356" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 37091 } } }
I20260807 22:46:42.015925 23847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:42.016310 23848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1722e4563eef485cac877a2c3414c356. Latest consensus state: current_term: 1 leader_uuid: "1722e4563eef485cac877a2c3414c356" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1722e4563eef485cac877a2c3414c356" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 37091 } } }
I20260807 22:46:42.016395 23848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356 [sys.catalog]: This master's current role is: LEADER
W20260807 22:46:42.019462 23865 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:46:42.019536 23865 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:46:42.019601 23856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:42.020267 23856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:42.025700 23856 catalog_manager.cc:1347] Generated new cluster ID: be327b5dff7b4a3fbb289b3f57b7a1be
I20260807 22:46:42.025753 23856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:42.051606 23856 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:42.052457 23856 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:42.056629 23856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: Generated new TSK 0
I20260807 22:46:42.057245 23856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:42.067036 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:37091
--builtin_ntp_servers=127.26.189.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:46:42.175870 23871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:42.176096 23871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:42.176131 23871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:42.180018 23871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:42.180063 23871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:42.180120 23871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:42.184676 23871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:42.185735 23871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:42.187045 23871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:42.195961 23879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.196003 23881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.195969 23878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:42.196473 23871 server_base.cc:1034] running on GCE node
I20260807 22:46:42.196831 23871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:42.197350 23871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:42.198524 23871 hybrid_clock.cc:648] HybridClock initialized: now 1786142802198474 us; error 30 us; skew 500 ppm
I20260807 22:46:42.200207 23871 webserver.cc:460] Webserver started at http://127.26.189.193:34347/ using document root <none> and password file <none>
I20260807 22:46:42.200714 23871 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:42.200771 23871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:42.200922 23871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:42.202841 23871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "be6d0ffdb8814331b50eb50fcb61d194"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "7a1f099bc51d3ed1e2500ef78b5af2d3"
server_key_iv: "d5175505de7df456f752dc8330bd1d62"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.203397 23871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "be6d0ffdb8814331b50eb50fcb61d194"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "7a1f099bc51d3ed1e2500ef78b5af2d3"
server_key_iv: "d5175505de7df456f752dc8330bd1d62"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.206840 23871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:42.209055 23891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.210135 23871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:42.210228 23871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "be6d0ffdb8814331b50eb50fcb61d194"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "7a1f099bc51d3ed1e2500ef78b5af2d3"
server_key_iv: "d5175505de7df456f752dc8330bd1d62"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.210337 23871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:42.230382 23871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:42.231006 23871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:42.231165 23871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:42.231726 23871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:42.232604 23871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:42.232651 23871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.232702 23871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:42.232726 23871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.241214 23871 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:46833
I20260807 22:46:42.241252 24055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:46833 every 8 connection(s)
I20260807 22:46:42.242040 23871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:42.244065 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 23871
I20260807 22:46:42.244185 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:42.244462 27383 external_mini_cluster.cc:1428] Setting key 503523b1ef3714fbc87a24dda170d8f9
I20260807 22:46:42.246387 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:37091
--builtin_ntp_servers=127.26.189.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:46:42.251859 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.243864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:50297 (local address 127.26.189.254:37091)
0807 22:46:42.244022 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.244027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.245134 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 22:46:42.246667 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.246670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.246719 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.246801 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:42.247691 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.248344 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.249925 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.250140 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.250213 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.251411 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:42.251414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:42.251417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:42.251429 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:42.251450 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:42.251459 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:42.251529 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.251685 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.251747 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:46:42.252946 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.244241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:37091 (local address 127.26.189.193:50297)
0807 22:46:42.244985 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.245037 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.246448 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.246994 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.247001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.247093 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:42.247553 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.247566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.248488 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.248493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.249821 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.249834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.250280 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.250284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.250498 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.251123 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:42.251125 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:42.251179 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:42.251259 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:42.251543 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:42.251552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:42.251558 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.251688 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.251983 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":137,"threads_started":1}
I20260807 22:46:42.254158 24057 heartbeater.cc:347] Connected to a master server at 127.26.189.254:37091
I20260807 22:46:42.254412 24057 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:42.254922 24057 heartbeater.cc:511] Master 127.26.189.254:37091 requested a full tablet report, sending...
I20260807 22:46:42.256434 23788 ts_manager.cc:194] Registered new tserver with Master: be6d0ffdb8814331b50eb50fcb61d194 (127.26.189.193:46833)
I20260807 22:46:42.257550 23788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.193:50297
W20260807 22:46:42.356184 24066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:42.356432 24066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:42.356493 24066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:42.360380 24066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:42.360424 24066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:42.360502 24066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:42.365091 24066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:42.366103 24066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:42.367422 24066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:42.376643 24074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.376657 24071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.376660 24072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:42.376801 24066 server_base.cc:1034] running on GCE node
I20260807 22:46:42.377305 24066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:42.377965 24066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:42.379124 24066 hybrid_clock.cc:648] HybridClock initialized: now 1786142802379107 us; error 34 us; skew 500 ppm
I20260807 22:46:42.383316 24066 webserver.cc:460] Webserver started at http://127.26.189.194:44559/ using document root <none> and password file <none>
I20260807 22:46:42.383857 24066 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:42.383921 24066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:42.384112 24066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:42.386142 24066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "f83838d407374879b52df38f6828a4f6"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "4de6e982ee0ffafe4fb189d9a86cc8ee"
server_key_iv: "78f455e993cd0117ad72a02707ebf50c"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.386734 24066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "f83838d407374879b52df38f6828a4f6"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "4de6e982ee0ffafe4fb189d9a86cc8ee"
server_key_iv: "78f455e993cd0117ad72a02707ebf50c"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.390125 24066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:46:42.392454 24083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.393597 24066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:42.393692 24066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "f83838d407374879b52df38f6828a4f6"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "4de6e982ee0ffafe4fb189d9a86cc8ee"
server_key_iv: "78f455e993cd0117ad72a02707ebf50c"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.393780 24066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:42.407563 24066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:42.408154 24066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:42.408309 24066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:42.408860 24066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:42.409747 24066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:42.409793 24066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.409832 24066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:42.409859 24066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.418654 24066 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:33043
I20260807 22:46:42.418679 24266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:33043 every 8 connection(s)
I20260807 22:46:42.419488 24066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:42.423615 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 24066
I20260807 22:46:42.423724 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:42.423987 27383 external_mini_cluster.cc:1428] Setting key 67ccc3a8c425d0d4659ba3f38246e2c4
I20260807 22:46:42.426041 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:37091
--builtin_ntp_servers=127.26.189.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:46:42.428751 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.421163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:42653 (local address 127.26.189.254:37091)
0807 22:46:42.421307 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.421311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.422439 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 22:46:42.423879 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.423882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.423944 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.424024 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:42.425039 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.425699 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.426936 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.427223 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.427335 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.428310 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:42.428314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:42.428317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:42.428329 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:42.428349 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:42.428356 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:42.428414 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.428553 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.428614 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:46:42.429531 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.421585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:37091 (local address 127.26.189.194:42653)
0807 22:46:42.422257 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.422331 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.423622 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.424199 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.424206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.424289 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:42.424887 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.424906 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.425835 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.425838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.426820 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.426830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.427364 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.427367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.427513 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.428068 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:42.428070 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:42.428107 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:42.428169 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:42.428426 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:42.428436 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:42.428441 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.428546 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.428810 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":123,"threads_started":1}
I20260807 22:46:42.430619 24267 heartbeater.cc:347] Connected to a master server at 127.26.189.254:37091
I20260807 22:46:42.430842 24267 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:42.431320 24267 heartbeater.cc:511] Master 127.26.189.254:37091 requested a full tablet report, sending...
I20260807 22:46:42.432327 23788 ts_manager.cc:194] Registered new tserver with Master: f83838d407374879b52df38f6828a4f6 (127.26.189.194:33043)
I20260807 22:46:42.432911 23788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.194:42653
W20260807 22:46:42.547324 24273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:42.547561 24273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:42.547612 24273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:42.551595 24273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:42.551643 24273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:42.551714 24273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:42.556290 24273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:42.557325 24273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:42.558733 24273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:46:42.567830 24273 server_base.cc:1034] running on GCE node
W20260807 22:46:42.567781 24280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.567787 24279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:42.567783 24284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:42.568375 24273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:42.568950 24273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:42.570107 24273 hybrid_clock.cc:648] HybridClock initialized: now 1786142802570089 us; error 30 us; skew 500 ppm
I20260807 22:46:42.571808 24273 webserver.cc:460] Webserver started at http://127.26.189.195:43831/ using document root <none> and password file <none>
I20260807 22:46:42.572319 24273 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:42.572384 24273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:42.572582 24273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:42.574574 24273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "31c416f14443470c96dda1901a1e69d6"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "5f1819bcfb771f0dd1fb4c4df11f8b57"
server_key_iv: "f898b3e1d4f8d208c6b135fd62d0717b"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.575084 24273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "31c416f14443470c96dda1901a1e69d6"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "5f1819bcfb771f0dd1fb4c4df11f8b57"
server_key_iv: "f898b3e1d4f8d208c6b135fd62d0717b"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.578387 24273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:46:42.580648 24294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.581722 24273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:42.581822 24273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "31c416f14443470c96dda1901a1e69d6"
format_stamp: "Formatted at 2026-08-07 22:46:42 on dist-test-slave-4tdj"
server_key: "5f1819bcfb771f0dd1fb4c4df11f8b57"
server_key_iv: "f898b3e1d4f8d208c6b135fd62d0717b"
server_key_version: "encryptionkey@0"
I20260807 22:46:42.581905 24273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:42.591925 24273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:42.592442 24273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:42.592597 24273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:42.593096 24273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:42.593935 24273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:42.593981 24273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.594028 24273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:42.594058 24273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:42.602159 24273 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:41831
I20260807 22:46:42.602190 24477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:41831 every 8 connection(s)
I20260807 22:46:42.602959 24273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:42.603662 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 24273
I20260807 22:46:42.603761 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:42.604012 27383 external_mini_cluster.cc:1428] Setting key 75323396d15d3527fbd16667db35a17d
I20260807 22:46:42.611621 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.604590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:45157 (local address 127.26.189.254:37091)
0807 22:46:42.604728 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.604733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.605885 (+ 1152us) server_negotiation.cc:407] Connection header received
0807 22:46:42.607312 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.607315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.607372 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.607455 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:42.608274 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.608796 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.609869 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.610089 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.610174 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.611179 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:42.611183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:42.611186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:42.611198 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:42.611216 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:42.611223 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:42.611280 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.611415 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.611481 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:46:42.612344 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.605029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:37091 (local address 127.26.189.195:45157)
0807 22:46:42.605751 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.605795 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.607115 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.607620 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.607627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.607705 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:42.608143 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.608155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.608912 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.608915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.609753 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.609762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.610219 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.610224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.610409 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.610938 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:42.610940 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:42.610983 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:42.611041 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:42.611290 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:42.611299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:42.611305 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.611406 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.611656 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":159,"threads_started":1}
I20260807 22:46:42.613318 24478 heartbeater.cc:347] Connected to a master server at 127.26.189.254:37091
I20260807 22:46:42.613507 24478 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:42.613934 24478 heartbeater.cc:511] Master 127.26.189.254:37091 requested a full tablet report, sending...
I20260807 22:46:42.614868 23788 ts_manager.cc:194] Registered new tserver with Master: 31c416f14443470c96dda1901a1e69d6 (127.26.189.195:41831)
I20260807 22:46:42.615423 23788 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.195:45157
I20260807 22:46:42.617774 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:42.623091 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.619600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.26.189.254:37091)
0807 22:46:42.619731 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.619735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.619845 (+ 110us) server_negotiation.cc:407] Connection header received
0807 22:46:42.619945 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.619947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.619984 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.620046 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:42.620340 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.620848 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.621674 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.621877 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.621954 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.622732 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:42.622735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:42.622737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:42.622748 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:42.622766 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:42.622773 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:42.622864 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.622939 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.622968 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:46:42.625602 23788 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:42.628484 23788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:42.641398 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.635863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33043 (local address 127.0.0.1:34276)
0807 22:46:42.636646 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.636692 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.636823 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.637389 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.637396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.637421 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:42.637598 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.637605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.638870 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.638873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.639837 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.639847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.640897 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.640900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.641015 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.641052 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.641155 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.641260 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":27,"thread_start_us":76,"threads_started":1}
I20260807 22:46:42.641444 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.636028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.26.189.194:33043)
0807 22:46:42.636311 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.636323 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.636966 (+ 643us) server_negotiation.cc:407] Connection header received
0807 22:46:42.637067 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.637072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.637238 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.637386 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:42.637714 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.638778 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.639970 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.640781 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.640899 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.641059 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.641185 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.641285 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
I20260807 22:46:42.642156 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.636484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:41831 (local address 127.0.0.1:53912)
0807 22:46:42.637390 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.637402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.637485 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.638094 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.638098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.638116 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:42.638194 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.638199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.639589 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.639593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.640606 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.640617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.641816 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.641819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.641889 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.641906 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.641970 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.642035 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":26,"thread_start_us":92,"threads_started":1}
I20260807 22:46:42.642467 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.636654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.26.189.195:41831)
0807 22:46:42.637067 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.637077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.637569 (+ 492us) server_negotiation.cc:407] Connection header received
0807 22:46:42.637641 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.637645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.637821 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.637953 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:42.638326 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.639447 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.640745 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.641477 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.641600 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.641755 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.642030 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.642121 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":131,"threads_started":1}
I20260807 22:46:42.642967 24500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.636723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:46833 (local address 127.0.0.1:34218)
0807 22:46:42.637850 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.637862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.638058 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.638499 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.638502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.638520 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:42.638592 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.638597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.639796 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.639799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.641128 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.641139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.642587 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.642591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.642689 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.642711 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.642785 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.642850 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":94,"threads_started":1}
I20260807 22:46:42.643671 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.636917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.26.189.193:46833)
0807 22:46:42.637230 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.637241 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.638004 (+ 763us) server_negotiation.cc:407] Connection header received
0807 22:46:42.638187 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.638192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.638370 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.638514 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:42.638678 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.639695 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.641261 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.642211 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.642357 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.642520 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.642855 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.642983 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":124,"threads_started":1}
I20260807 22:46:42.644712 24164 tablet_service.cc:1467] Processing CreateTablet for tablet e11bd93ffcaf4e4da565ca53addb3dc7 (DEFAULT_TABLE table=test-table [id=9e66730e73a4493c85266cd603da8499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:42.645956 24164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11bd93ffcaf4e4da565ca53addb3dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:42.645704 24370 tablet_service.cc:1467] Processing CreateTablet for tablet e11bd93ffcaf4e4da565ca53addb3dc7 (DEFAULT_TABLE table=test-table [id=9e66730e73a4493c85266cd603da8499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:42.646646 24370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11bd93ffcaf4e4da565ca53addb3dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:42.652089 23964 tablet_service.cc:1467] Processing CreateTablet for tablet e11bd93ffcaf4e4da565ca53addb3dc7 (DEFAULT_TABLE table=test-table [id=9e66730e73a4493c85266cd603da8499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:42.653028 24502 tablet_bootstrap.cc:492] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Bootstrap starting.
I20260807 22:46:42.653159 23964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11bd93ffcaf4e4da565ca53addb3dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:42.655354 24502 tablet_bootstrap.cc:654] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:42.657088 24502 log.cc:824] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:42.658839 24506 tablet_bootstrap.cc:492] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Bootstrap starting.
I20260807 22:46:42.660326 24507 tablet_bootstrap.cc:492] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: Bootstrap starting.
I20260807 22:46:42.660353 24502 tablet_bootstrap.cc:492] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: No bootstrap required, opened a new log
I20260807 22:46:42.660650 24502 ts_tablet_manager.cc:1397] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 22:46:42.661159 24506 tablet_bootstrap.cc:654] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:42.662160 24506 log.cc:824] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:42.662204 24507 tablet_bootstrap.cc:654] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:42.662982 24507 log.cc:824] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:42.664665 24502 raft_consensus.cc:348] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.664884 24502 raft_consensus.cc:374] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:42.664921 24502 raft_consensus.cc:729] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f83838d407374879b52df38f6828a4f6, State: Initialized, Role: FOLLOWER
I20260807 22:46:42.665374 24507 tablet_bootstrap.cc:492] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: No bootstrap required, opened a new log
I20260807 22:46:42.665345 24502 consensus_queue.cc:260] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.665550 24507 ts_tablet_manager.cc:1397] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 22:46:42.665967 24267 heartbeater.cc:503] Master 127.26.189.254:37091 was elected leader, sending a full tablet report...
I20260807 22:46:42.666273 24502 ts_tablet_manager.cc:1428] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 22:46:42.667838 24506 tablet_bootstrap.cc:492] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: No bootstrap required, opened a new log
I20260807 22:46:42.667971 24506 ts_tablet_manager.cc:1397] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 22:46:42.668877 24507 raft_consensus.cc:348] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.669128 24507 raft_consensus.cc:374] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:42.669185 24507 raft_consensus.cc:729] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6d0ffdb8814331b50eb50fcb61d194, State: Initialized, Role: FOLLOWER
I20260807 22:46:42.669621 24507 consensus_queue.cc:260] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.670238 24057 heartbeater.cc:503] Master 127.26.189.254:37091 was elected leader, sending a full tablet report...
W20260807 22:46:42.670356 24270 tablet.cc:2369] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:42.670596 24507 ts_tablet_manager.cc:1428] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260807 22:46:42.672046 24506 raft_consensus.cc:348] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.672238 24506 raft_consensus.cc:374] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:42.672274 24506 raft_consensus.cc:729] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c416f14443470c96dda1901a1e69d6, State: Initialized, Role: FOLLOWER
I20260807 22:46:42.672652 24506 consensus_queue.cc:260] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.673323 24478 heartbeater.cc:503] Master 127.26.189.254:37091 was elected leader, sending a full tablet report...
I20260807 22:46:42.673488 24506 ts_tablet_manager.cc:1428] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 22:46:42.743139 24063 tablet.cc:2369] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:46:42.853904 24479 tablet.cc:2369] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:42.926592 24511 raft_consensus.cc:482] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:42.926860 24511 raft_consensus.cc:504] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.928130 24511 leader_election.cc:288] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f83838d407374879b52df38f6828a4f6 (127.26.189.194:33043), 31c416f14443470c96dda1901a1e69d6 (127.26.189.195:41831)
I20260807 22:46:42.931990 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.928509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:53341 (local address 127.26.189.194:33043)
0807 22:46:42.928663 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.928668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.928801 (+ 133us) server_negotiation.cc:407] Connection header received
0807 22:46:42.929079 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.929082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.929137 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.929209 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:42.929511 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.930016 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.931110 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.931490 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.931599 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.931654 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.931810 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.931881 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:46:42.932008 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.928387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:33043 (local address 127.26.189.193:53341)
0807 22:46:42.928719 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.928733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.928900 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.929246 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.929248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.929267 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:42.929390 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.929397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.930136 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.930139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.930979 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.930989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.931608 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.931612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.931694 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.931717 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.931782 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.931849 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":88,"threads_started":1}
I20260807 22:46:42.932204 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.929011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:49863 (local address 127.26.189.195:41831)
0807 22:46:42.929146 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:42.929150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:42.929192 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:46:42.929333 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:42.929336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:42.929376 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:42.929437 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:42.929692 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.930273 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.931276 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.931677 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.931783 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.931829 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:42.931995 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 22:46:42.932073 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:46:42.932526 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:42.928763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:41831 (local address 127.26.189.193:49863)
0807 22:46:42.929121 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:42.929134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:42.929223 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:42.929456 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:42.929458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:42.929473 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:42.929594 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.929599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.930400 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.930403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.931164 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:42.931173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:42.931797 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:42.931801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:42.931890 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:42.931908 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:46:42.931970 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 22:46:42.932037 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":82,"threads_started":1}
I20260807 22:46:42.932615 24193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11bd93ffcaf4e4da565ca53addb3dc7" candidate_uuid: "be6d0ffdb8814331b50eb50fcb61d194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f83838d407374879b52df38f6828a4f6" is_pre_election: true
I20260807 22:46:42.932914 24193 raft_consensus.cc:2454] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6d0ffdb8814331b50eb50fcb61d194 in term 0.
I20260807 22:46:42.933092 24394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11bd93ffcaf4e4da565ca53addb3dc7" candidate_uuid: "be6d0ffdb8814331b50eb50fcb61d194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c416f14443470c96dda1901a1e69d6" is_pre_election: true
I20260807 22:46:42.933365 24394 raft_consensus.cc:2454] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6d0ffdb8814331b50eb50fcb61d194 in term 0.
I20260807 22:46:42.933375 23893 leader_election.cc:302] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6d0ffdb8814331b50eb50fcb61d194, f83838d407374879b52df38f6828a4f6; no voters:
I20260807 22:46:42.933624 24511 raft_consensus.cc:2781] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:42.933706 24511 raft_consensus.cc:482] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:42.933744 24511 raft_consensus.cc:3037] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:42.934845 24511 raft_consensus.cc:504] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.935155 24511 leader_election.cc:288] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [CANDIDATE]: Term 1 election: Requested vote from peers f83838d407374879b52df38f6828a4f6 (127.26.189.194:33043), 31c416f14443470c96dda1901a1e69d6 (127.26.189.195:41831)
I20260807 22:46:42.935487 24193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11bd93ffcaf4e4da565ca53addb3dc7" candidate_uuid: "be6d0ffdb8814331b50eb50fcb61d194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f83838d407374879b52df38f6828a4f6"
I20260807 22:46:42.935561 24394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11bd93ffcaf4e4da565ca53addb3dc7" candidate_uuid: "be6d0ffdb8814331b50eb50fcb61d194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c416f14443470c96dda1901a1e69d6"
I20260807 22:46:42.935603 24193 raft_consensus.cc:3037] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:42.935649 24394 raft_consensus.cc:3037] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:42.936574 24394 raft_consensus.cc:2454] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6d0ffdb8814331b50eb50fcb61d194 in term 1.
I20260807 22:46:42.936651 24193 raft_consensus.cc:2454] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6d0ffdb8814331b50eb50fcb61d194 in term 1.
I20260807 22:46:42.936858 23893 leader_election.cc:302] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c416f14443470c96dda1901a1e69d6, be6d0ffdb8814331b50eb50fcb61d194; no voters:
I20260807 22:46:42.937060 24511 raft_consensus.cc:2781] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:42.937260 24511 raft_consensus.cc:686] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [term 1 LEADER]: Becoming Leader. State: Replica: be6d0ffdb8814331b50eb50fcb61d194, State: Running, Role: LEADER
I20260807 22:46:42.937556 24511 consensus_queue.cc:237] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } }
I20260807 22:46:42.940773 23788 catalog_manager.cc:5495] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 reported cstate change: term changed from 0 to 1, leader changed from <none> to be6d0ffdb8814331b50eb50fcb61d194 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "be6d0ffdb8814331b50eb50fcb61d194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be6d0ffdb8814331b50eb50fcb61d194" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 46833 } health_report { overall_health: HEALTHY } } }
I20260807 22:46:43.007899 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.004571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.26.189.193:46833)
0807 22:46:43.004687 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:43.004691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:43.004705 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:46:43.004843 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:43.004845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:43.004915 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:43.005051 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:43.005299 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.005794 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.006670 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:43.006894 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:43.006981 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:43.007214 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:43.007575 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:43.007680 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:43.007759 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 22:46:43.007791 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:46:43.013440 24193 raft_consensus.cc:1261] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 1 FOLLOWER]: Refusing update from remote peer be6d0ffdb8814331b50eb50fcb61d194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:43.013453 24394 raft_consensus.cc:1261] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 1 FOLLOWER]: Refusing update from remote peer be6d0ffdb8814331b50eb50fcb61d194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:43.014099 24511 consensus_queue.cc:1035] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31c416f14443470c96dda1901a1e69d6" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:43.014286 24516 consensus_queue.cc:1035] T e11bd93ffcaf4e4da565ca53addb3dc7 P be6d0ffdb8814331b50eb50fcb61d194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f83838d407374879b52df38f6828a4f6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 33043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:43.023716 24522 mvcc.cc:205] Tried to move back new op lower bound from 7316040921136267264 to 7316040920834699264. Current Snapshot: MvccSnapshot[applied={T|T < 7316040921136267264}]
I20260807 22:46:43.023880 24523 mvcc.cc:205] Tried to move back new op lower bound from 7316040921136267264 to 7316040920834699264. Current Snapshot: MvccSnapshot[applied={T|T < 7316040921136267264}]
I20260807 22:46:43.024614 24524 mvcc.cc:205] Tried to move back new op lower bound from 7316040921136267264 to 7316040920834699264. Current Snapshot: MvccSnapshot[applied={T|T < 7316040921136267264}]
I20260807 22:46:43.035213 23788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:43.035362 23788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:43.037561 23788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1722e4563eef485cac877a2c3414c356: Sending DeleteTablet for 3 replicas of tablet e11bd93ffcaf4e4da565ca53addb3dc7
I20260807 22:46:43.038264 24164 tablet_service.cc:1514] Processing DeleteTablet for tablet e11bd93ffcaf4e4da565ca53addb3dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:43 UTC) from {username='slave'} at 127.0.0.1:34276
I20260807 22:46:43.038337 23964 tablet_service.cc:1514] Processing DeleteTablet for tablet e11bd93ffcaf4e4da565ca53addb3dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:43 UTC) from {username='slave'} at 127.0.0.1:34218
I20260807 22:46:43.038399 24370 tablet_service.cc:1514] Processing DeleteTablet for tablet e11bd93ffcaf4e4da565ca53addb3dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:43 UTC) from {username='slave'} at 127.0.0.1:53912
I20260807 22:46:43.038906 24537 tablet_replica.cc:330] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: stopping tablet replica
I20260807 22:46:43.039232 24537 raft_consensus.cc:2229] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:43.039551 24537 raft_consensus.cc:2258] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:43.039618 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 23871
I20260807 22:46:43.040284 24539 tablet_replica.cc:330] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: stopping tablet replica
I20260807 22:46:43.040650 24539 raft_consensus.cc:2229] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:43.040983 24539 raft_consensus.cc:2258] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:43.046229 24539 ts_tablet_manager.cc:1905] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:43.048175 24537 ts_tablet_manager.cc:1905] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:43.049237 24539 ts_tablet_manager.cc:1918] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:43.049335 24539 log.cc:1196] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/e11bd93ffcaf4e4da565ca53addb3dc7
I20260807 22:46:43.049621 24539 ts_tablet_manager.cc:1939] T e11bd93ffcaf4e4da565ca53addb3dc7 P 31c416f14443470c96dda1901a1e69d6: Deleting consensus metadata
I20260807 22:46:43.050495 24537 ts_tablet_manager.cc:1918] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:43.050643 24537 log.cc:1196] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/e11bd93ffcaf4e4da565ca53addb3dc7
I20260807 22:46:43.050859 23765 catalog_manager.cc:4841] TS 31c416f14443470c96dda1901a1e69d6 (127.26.189.195:41831): tablet e11bd93ffcaf4e4da565ca53addb3dc7 (table test-table [id=9e66730e73a4493c85266cd603da8499]) successfully deleted
I20260807 22:46:43.050947 24537 ts_tablet_manager.cc:1939] T e11bd93ffcaf4e4da565ca53addb3dc7 P f83838d407374879b52df38f6828a4f6: Deleting consensus metadata
I20260807 22:46:43.051893 23765 catalog_manager.cc:4841] TS f83838d407374879b52df38f6828a4f6 (127.26.189.194:33043): tablet e11bd93ffcaf4e4da565ca53addb3dc7 (table test-table [id=9e66730e73a4493c85266cd603da8499]) successfully deleted
W20260807 22:46:43.053884 23766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:46833 (error 108)
I20260807 22:46:43.054706 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 24066
I20260807 22:46:43.054960 24500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:43.054668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:46833 (local address 127.0.0.1:34232)
0807 22:46:43.054808 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:43.054878 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:46833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 22:46:43.067955 23766 catalog_manager.cc:4584] TS be6d0ffdb8814331b50eb50fcb61d194 (127.26.189.193:46833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e11bd93ffcaf4e4da565ca53addb3dc7: Network error: Client connection negotiation failed: client connection to 127.26.189.193:46833: connect: Connection refused (error 111)
I20260807 22:46:43.072804 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 24273
I20260807 22:46:43.084625 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 23744
2026-08-07T22:46:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1314 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:43 dist-test-slave-4tdj krb5kdc[24545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:43 dist-test-slave-4tdj krb5kdc[24545](info): set up 2 sockets
Aug 07 22:46:43 dist-test-slave-4tdj krb5kdc[24545](info): commencing operation
krb5kdc: starting...
W20260807 22:46:44.214506 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:44 dist-test-slave-4tdj krb5kdc[24545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:46:44.333777 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39149
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:44.441517 24561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:44.441769 24561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:44.441812 24561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:44.445776 24561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:44.445828 24561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:44.445845 24561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:44.445861 24561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:44.445891 24561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:44.451035 24561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:39149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:44.451989 24561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:44.452953 24561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:44.461344 24573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:44.461367 24574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:44.461349 24577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:44.461581 24561 server_base.cc:1034] running on GCE node
I20260807 22:46:44.462167 24561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:44.463195 24561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:44.464350 24561 hybrid_clock.cc:648] HybridClock initialized: now 1786142804464334 us; error 31 us; skew 500 ppm
Aug 07 22:46:44 dist-test-slave-4tdj krb5kdc[24545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:44.466964 24561 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:46:44.467800 24561 webserver.cc:460] Webserver started at http://127.26.189.254:35587/ using document root <none> and password file <none>
I20260807 22:46:44.468325 24561 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:44.468392 24561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:44.468583 24561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:44.470620 24561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "b15db500a7614487ae3de71eecfe1af7"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "2c73801da1c74f434aa39fb5e673388f"
server_key_iv: "9b14ebd70c1b9d5b78c9b15e441c4de6"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.471158 24561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "b15db500a7614487ae3de71eecfe1af7"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "2c73801da1c74f434aa39fb5e673388f"
server_key_iv: "9b14ebd70c1b9d5b78c9b15e441c4de6"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.474613 24561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:46:44.476795 24585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:44.477914 24561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:44.478024 24561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "b15db500a7614487ae3de71eecfe1af7"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "2c73801da1c74f434aa39fb5e673388f"
server_key_iv: "9b14ebd70c1b9d5b78c9b15e441c4de6"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.478124 24561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:44.502504 24561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:44.505533 24561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:44.505724 24561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:44.512763 24561 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:39149
I20260807 22:46:44.512787 24671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:39149 every 8 connection(s)
I20260807 22:46:44.513659 24561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:44.516458 24672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:44.519807 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 24561
I20260807 22:46:44.519950 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:44.520282 27383 external_mini_cluster.cc:1428] Setting key 0659aa378bed65696089b59fcc5912a5
I20260807 22:46:44.522644 24672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: Bootstrap starting.
I20260807 22:46:44.524806 24672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:44.525573 24672 log.cc:824] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:44.527374 24672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: No bootstrap required, opened a new log
Aug 07 22:46:44 dist-test-slave-4tdj krb5kdc[24545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:44.530987 24672 raft_consensus.cc:348] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15db500a7614487ae3de71eecfe1af7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39149 } }
I20260807 22:46:44.531194 24672 raft_consensus.cc:374] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:44.531251 24672 raft_consensus.cc:729] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15db500a7614487ae3de71eecfe1af7, State: Initialized, Role: FOLLOWER
I20260807 22:46:44.531689 24672 consensus_queue.cc:260] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15db500a7614487ae3de71eecfe1af7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39149 } }
I20260807 22:46:44.531848 24672 raft_consensus.cc:388] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:44.531919 24672 raft_consensus.cc:482] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:44.532025 24672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:44.532868 24672 raft_consensus.cc:504] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15db500a7614487ae3de71eecfe1af7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39149 } }
I20260807 22:46:44.533196 24672 leader_election.cc:302] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b15db500a7614487ae3de71eecfe1af7; no voters:
I20260807 22:46:44.533479 24672 leader_election.cc:288] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:44.533599 24681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:44.533833 24681 raft_consensus.cc:686] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [term 1 LEADER]: Becoming Leader. State: Replica: b15db500a7614487ae3de71eecfe1af7, State: Running, Role: LEADER
I20260807 22:46:44.534202 24681 consensus_queue.cc:237] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15db500a7614487ae3de71eecfe1af7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39149 } }
I20260807 22:46:44.534355 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.521775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.26.189.254:39149)
0807 22:46:44.522200 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.522214 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.522249 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:46:44.523010 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.523030 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.523344 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.523646 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.524444 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.525520 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.526433 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.526887 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.527030 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.529512 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.529517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.529530 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.529556 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.531115 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.531594 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.531600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.531606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.531749 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.532045 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.532048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.532049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.532464 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.532670 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.532923 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.533308 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":287,"thread_start_us":135,"threads_started":1}
I20260807 22:46:44.534734 24672 sys_catalog.cc:564] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:44.537225 24683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b15db500a7614487ae3de71eecfe1af7. Latest consensus state: current_term: 1 leader_uuid: "b15db500a7614487ae3de71eecfe1af7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15db500a7614487ae3de71eecfe1af7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39149 } } }
I20260807 22:46:44.537329 24683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:44.537719 24682 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b15db500a7614487ae3de71eecfe1af7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15db500a7614487ae3de71eecfe1af7" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 39149 } } }
I20260807 22:46:44.538030 24682 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7 [sys.catalog]: This master's current role is: LEADER
W20260807 22:46:44.540215 24705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:46:44.540297 24705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:46:44.540369 24704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:44.541249 24704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:44.546419 24704 catalog_manager.cc:1347] Generated new cluster ID: 270b56af527d45dbb8304f97586cf6b2
I20260807 22:46:44.546744 24704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:44.561815 24704 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:44.562979 24704 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:44.576247 24704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: Generated new TSK 0
I20260807 22:46:44.576939 24704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.189.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
Entry for principal kudu/127.26.189.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab.
I20260807 22:46:44.604913 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39149
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:44.713199 24712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:44.713433 24712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:44.713472 24712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:44.717365 24712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:44.717415 24712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:44.717473 24712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:44.722563 24712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.193.keytab
--principal=kudu/127.26.189.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:44.723537 24712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:44.724488 24712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:44.733505 24718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:44.733489 24721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:44.733714 24712 server_base.cc:1034] running on GCE node
W20260807 22:46:44.733820 24719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:44.734242 24712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:44.734913 24712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:44.736130 24712 hybrid_clock.cc:648] HybridClock initialized: now 1786142804736114 us; error 30 us; skew 500 ppm
Aug 07 22:46:44 dist-test-slave-4tdj krb5kdc[24545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:44.738760 24712 init.cc:377] Logged in from keytab as kudu/127.26.189.193@KRBTEST.COM (short username kudu)
I20260807 22:46:44.739512 24712 webserver.cc:460] Webserver started at http://127.26.189.193:37659/ using document root <none> and password file <none>
I20260807 22:46:44.739992 24712 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:44.740056 24712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:44.740247 24712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:44.742162 24712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "50aaacf869cb6f8c9894b97a8acd1f79"
server_key_iv: "b9dba570b09e96dce100c8e40fd0d4ba"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.742734 24712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "50aaacf869cb6f8c9894b97a8acd1f79"
server_key_iv: "b9dba570b09e96dce100c8e40fd0d4ba"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.746218 24712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:44.748495 24734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:44.749620 24712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:44.749722 24712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "50aaacf869cb6f8c9894b97a8acd1f79"
server_key_iv: "b9dba570b09e96dce100c8e40fd0d4ba"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.749809 24712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:44.779277 24712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:44.782533 24712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:44.782717 24712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:44.783306 24712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:44.784133 24712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:44.784184 24712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:44.784237 24712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:44.784266 24712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:44.792587 24712 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:40557
I20260807 22:46:44.792613 24900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:40557 every 8 connection(s)
I20260807 22:46:44.793430 24712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:44.801119 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 24712
I20260807 22:46:44.801252 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:44.801533 27383 external_mini_cluster.cc:1428] Setting key 7a8086d243e145a6b2be9350a0e73553
Aug 07 22:46:44 dist-test-slave-4tdj krb5kdc[24545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.193@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:44.806603 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.795140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:44441 (local address 127.26.189.254:39149)
0807 22:46:44.795297 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:44.795302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:44.796107 (+ 805us) server_negotiation.cc:407] Connection header received
0807 22:46:44.797324 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:44.797326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:44.797374 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:44.797446 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:44.798518 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.799037 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.800120 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.800342 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.800416 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.802918 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:44.802922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:44.802925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:44.802953 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:44.804468 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.804956 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.804960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.804961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.805060 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:44.805499 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:44.805501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:44.805503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:44.805681 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:44.805754 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:44.806378 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 22:46:44.806485 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:46:44.807615 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:44.795519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39149 (local address 127.26.189.193:44441)
0807 22:46:44.795955 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:44.795999 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:44.797140 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:44.797597 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:44.797604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:44.797915 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:44.798409 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:44.798425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.799146 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.799149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:44.800020 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:44.800029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:44.800465 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:44.800469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:44.800626 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:44.801202 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:44.801218 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:44.802775 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:44.804591 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.804596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.804608 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.804862 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.805190 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:44.805194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:44.805196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:44.805396 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:44.805779 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:44.805785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:44.806099 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 22:46:44.806344 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 22:46:44.806667 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260807 22:46:44.808847 24901 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39149
I20260807 22:46:44.809074 24901 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:44.809525 24901 heartbeater.cc:511] Master 127.26.189.254:39149 requested a full tablet report, sending...
I20260807 22:46:44.811151 24611 ts_manager.cc:194] Registered new tserver with Master: cf46c922aacd4adeb57cf0fc1ddc9b74 (127.26.189.193:40557)
WARNING: no policy specified for kudu/127.26.189.194@KRBTEST.COM; defaulting to no policy
I20260807 22:46:44.812711 24611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.193@KRBTEST.COM'} at 127.26.189.193:44441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
Entry for principal kudu/127.26.189.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab.
I20260807 22:46:44.829345 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39149
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:44.937589 24909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:44.937816 24909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:44.937856 24909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:44.941747 24909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:44.941797 24909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:44.941869 24909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:44.946883 24909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.194.keytab
--principal=kudu/127.26.189.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:44.947765 24909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:44.948665 24909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:44.957733 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:44.957741 24914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:44.957856 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:44.958029 24909 server_base.cc:1034] running on GCE node
I20260807 22:46:44.958454 24909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:44.959069 24909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:44.960250 24909 hybrid_clock.cc:648] HybridClock initialized: now 1786142804960222 us; error 40 us; skew 500 ppm
Aug 07 22:46:44 dist-test-slave-4tdj krb5kdc[24545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:44.962980 24909 init.cc:377] Logged in from keytab as kudu/127.26.189.194@KRBTEST.COM (short username kudu)
I20260807 22:46:44.963796 24909 webserver.cc:460] Webserver started at http://127.26.189.194:40991/ using document root <none> and password file <none>
I20260807 22:46:44.964319 24909 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:44.964385 24909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:44.964578 24909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:44.966578 24909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "9827af2bab1a4e988724add08063fba6"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "264f375257129c93a59bd8b7d5252d4f"
server_key_iv: "63e2a115ff5ac5f5539cfdb04997519a"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.967098 24909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "9827af2bab1a4e988724add08063fba6"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "264f375257129c93a59bd8b7d5252d4f"
server_key_iv: "63e2a115ff5ac5f5539cfdb04997519a"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.970522 24909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 22:46:44.972697 24931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:44.973814 24909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:44.973930 24909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "9827af2bab1a4e988724add08063fba6"
format_stamp: "Formatted at 2026-08-07 22:46:44 on dist-test-slave-4tdj"
server_key: "264f375257129c93a59bd8b7d5252d4f"
server_key_iv: "63e2a115ff5ac5f5539cfdb04997519a"
server_key_version: "encryptionkey@0"
I20260807 22:46:44.974028 24909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:44.993266 24909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:44.996222 24909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:44.996389 24909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:44.996935 24909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:44.997720 24909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:44.997766 24909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:44.997817 24909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:44.997845 24909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.006357 24909 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:35581
I20260807 22:46:45.006385 25092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:35581 every 8 connection(s)
I20260807 22:46:45.007230 24909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:45.015210 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 24909
I20260807 22:46:45.015333 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:45.015614 27383 external_mini_cluster.cc:1428] Setting key 0c651d787d38b6b98fb1f29dff0f0765
Aug 07 22:46:45 dist-test-slave-4tdj krb5kdc[24545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142804, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.194@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:45.020229 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.008997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:56927 (local address 127.26.189.254:39149)
0807 22:46:45.009146 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.009150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.009968 (+ 818us) server_negotiation.cc:407] Connection header received
0807 22:46:45.011298 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.011301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.011349 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.011424 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:45.012532 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.013061 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.014175 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.014456 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.014545 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.016982 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:45.016986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:45.016989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:45.017013 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:45.018259 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:45.018742 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:45.018745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:45.018747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:45.018824 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:45.019162 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:45.019166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:45.019167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:45.019397 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:45.019502 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.019948 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.020073 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:46:45.021128 25095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.009371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39149 (local address 127.26.189.194:56927)
0807 22:46:45.009824 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.009866 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.011091 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.011583 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.011589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.011926 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:45.012367 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.012380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.013199 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.013203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.014058 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.014067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.014596 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.014599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.014730 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.015246 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:45.015264 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:45.016844 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:45.018391 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:45.018396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:45.018410 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:45.018651 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:45.018906 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:45.018909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:45.018911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:45.019074 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:45.019504 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:45.019510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:45.019734 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.019891 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.020197 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260807 22:46:45.023562 25093 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39149
I20260807 22:46:45.023815 25093 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:45.024379 25093 heartbeater.cc:511] Master 127.26.189.254:39149 requested a full tablet report, sending...
I20260807 22:46:45.025487 24611 ts_manager.cc:194] Registered new tserver with Master: 9827af2bab1a4e988724add08063fba6 (127.26.189.194:35581)
I20260807 22:46:45.026238 24611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.194@KRBTEST.COM'} at 127.26.189.194:56927
WARNING: no policy specified for kudu/127.26.189.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
Entry for principal kudu/127.26.189.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab.
I20260807 22:46:45.045712 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39149
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:45.155249 25099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:45.155475 25099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:45.155526 25099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:45.159421 25099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:45.159472 25099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:45.159544 25099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:45.164703 25099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.195.keytab
--principal=kudu/127.26.189.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:39149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:45.165766 25099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:45.166801 25099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:45.176002 25106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:45.176017 25105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:45.176049 25108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:45.176111 25099 server_base.cc:1034] running on GCE node
I20260807 22:46:45.176622 25099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:45.177255 25099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:45.178407 25099 hybrid_clock.cc:648] HybridClock initialized: now 1786142805178363 us; error 54 us; skew 500 ppm
Aug 07 22:46:45 dist-test-slave-4tdj krb5kdc[24545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142805, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:45.181164 25099 init.cc:377] Logged in from keytab as kudu/127.26.189.195@KRBTEST.COM (short username kudu)
I20260807 22:46:45.182096 25099 webserver.cc:460] Webserver started at http://127.26.189.195:45147/ using document root <none> and password file <none>
I20260807 22:46:45.182695 25099 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:45.182767 25099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:45.182965 25099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:45.184949 25099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "9e71076886db40df89ab0940d9cee2a0"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "47a9fe80da23f42a49c3ad91e9f547e1"
server_key_iv: "2832ab07c756a74b6cdbe584a1bc4d37"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.185514 25099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "9e71076886db40df89ab0940d9cee2a0"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "47a9fe80da23f42a49c3ad91e9f547e1"
server_key_iv: "2832ab07c756a74b6cdbe584a1bc4d37"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.189137 25099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:46:45.191610 25116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.192802 25099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:46:45.192925 25099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "9e71076886db40df89ab0940d9cee2a0"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "47a9fe80da23f42a49c3ad91e9f547e1"
server_key_iv: "2832ab07c756a74b6cdbe584a1bc4d37"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.193014 25099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:45.201951 25099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:45.204870 25099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:45.205031 25099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:45.205588 25099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:45.206451 25099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:45.206501 25099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.206552 25099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:45.206585 25099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.214982 25099 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:36205
I20260807 22:46:45.215009 25269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:36205 every 8 connection(s)
I20260807 22:46:45.215814 25099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:45.222357 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 25099
I20260807 22:46:45.222528 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:45.222829 27383 external_mini_cluster.cc:1428] Setting key 6d83d4aaf009de0063e987bbc3df6dcb
Aug 07 22:46:45 dist-test-slave-4tdj krb5kdc[24545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142805, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.195@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:46:45.229209 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.217531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:39837 (local address 127.26.189.254:39149)
0807 22:46:45.217670 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.217674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.218543 (+ 869us) server_negotiation.cc:407] Connection header received
0807 22:46:45.219837 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.219841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.219901 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.219976 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:45.221146 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.221677 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.223181 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.223464 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.223611 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.226223 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:45.226226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:45.226228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:45.226254 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:45.227580 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:45.227996 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:45.227998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:45.227999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:45.228071 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:45.228315 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:45.228317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:45.228318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:45.228473 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:45.228581 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.228995 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.229095 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:46:45.229995 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.217905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:39149 (local address 127.26.189.195:39837)
0807 22:46:45.218372 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.218416 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.219630 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.220168 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.220175 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.220531 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:45.220992 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.221005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.221796 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.221800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.223029 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.223060 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.223606 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.223609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.223807 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.224426 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:46:45.224447 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:45.226064 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:45.227676 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:45.227681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:45.227693 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:45.227924 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:45.228134 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:46:45.228136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:46:45.228137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:46:45.228253 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:46:45.228577 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:45.228582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:45.228796 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.228937 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.229167 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
I20260807 22:46:45.231040 25270 heartbeater.cc:347] Connected to a master server at 127.26.189.254:39149
I20260807 22:46:45.231240 25270 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:45.231665 25270 heartbeater.cc:511] Master 127.26.189.254:39149 requested a full tablet report, sending...
I20260807 22:46:45.232577 24611 ts_manager.cc:194] Registered new tserver with Master: 9e71076886db40df89ab0940d9cee2a0 (127.26.189.195:36205)
I20260807 22:46:45.233119 24611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.189.195@KRBTEST.COM'} at 127.26.189.195:39837
I20260807 22:46:45.236472 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:45.244722 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.238344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.26.189.254:39149)
0807 22:46:45.238462 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.238466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.238637 (+ 171us) server_negotiation.cc:407] Connection header received
0807 22:46:45.238720 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.238723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.238758 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.238816 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:45.239398 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.239865 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.240606 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.240797 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.240873 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.242130 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:45.242133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:45.242135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:45.242156 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:45.243238 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:45.243613 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:45.243617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:45.243619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:45.243691 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:45.243929 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:45.243932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:45.243934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:45.244144 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:45.244260 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.244479 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.244585 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:46:45.247265 24611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:45.250243 24611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:45.263726 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.258059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36205 (local address 127.0.0.1:57040)
0807 22:46:45.258655 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.258705 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.258820 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.259515 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.259521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.259558 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:45.259678 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.259685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.261070 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.261074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.262262 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.262271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.263208 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.263212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.263318 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.263356 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.263461 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.263577 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":30,"thread_start_us":84,"threads_started":1}
I20260807 22:46:45.263739 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.258205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.26.189.195:36205)
0807 22:46:45.258878 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.258889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.258912 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:46:45.258990 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.258995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.259195 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.259345 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:45.259817 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.260942 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.262386 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.263092 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.263214 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.263340 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.263500 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.263602 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":189,"threads_started":1}
I20260807 22:46:45.264005 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.258132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:35581 (local address 127.0.0.1:35628)
0807 22:46:45.259412 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.259425 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.259512 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.260157 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.260160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.260176 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:45.260260 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.260267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.261353 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.261356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.262612 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.262623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.263656 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.263660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.263746 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.263769 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.263820 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.263885 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1222,"mutex_wait_us":21,"thread_start_us":115,"threads_started":1}
I20260807 22:46:45.264400 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.259437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:40557 (local address 127.0.0.1:49578)
0807 22:46:45.259696 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.259709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.259785 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.260384 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.260386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.260404 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:45.260478 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.260484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.261815 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.261818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.262877 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.262885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.264037 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.264041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.264127 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.264148 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.264213 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.264279 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":98,"threads_started":1}
I20260807 22:46:45.264465 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.258858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.26.189.194:35581)
0807 22:46:45.259787 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.259798 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.259820 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:46:45.259892 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.259898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.260031 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.260158 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:45.260363 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.261213 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.262735 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.263510 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.264101 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.264195 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.264245 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.264334 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":113,"threads_started":1}
I20260807 22:46:45.264874 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.259605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.26.189.193:40557)
0807 22:46:45.259981 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.259995 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.260017 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:46:45.260100 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.260105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.260271 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.260400 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:45.260566 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.261667 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.262981 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.263739 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.263856 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.263975 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.264653 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.264747 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":111,"threads_started":1}
I20260807 22:46:45.267171 25187 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6320f15cdc4f45a8f386fe9721d328 (DEFAULT_TABLE table=test-table [id=d495369534294a828600f54d90a469ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:45.267198 24804 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6320f15cdc4f45a8f386fe9721d328 (DEFAULT_TABLE table=test-table [id=d495369534294a828600f54d90a469ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:45.267198 25000 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6320f15cdc4f45a8f386fe9721d328 (DEFAULT_TABLE table=test-table [id=d495369534294a828600f54d90a469ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:45.268054 25187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6320f15cdc4f45a8f386fe9721d328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:45.268184 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6320f15cdc4f45a8f386fe9721d328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:45.268184 24804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6320f15cdc4f45a8f386fe9721d328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:45.274572 25296 tablet_bootstrap.cc:492] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Bootstrap starting.
I20260807 22:46:45.275300 25297 tablet_bootstrap.cc:492] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Bootstrap starting.
I20260807 22:46:45.275647 25299 tablet_bootstrap.cc:492] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: Bootstrap starting.
I20260807 22:46:45.277132 25296 tablet_bootstrap.cc:654] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:45.277446 25299 tablet_bootstrap.cc:654] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:45.277451 25297 tablet_bootstrap.cc:654] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:45.278546 25299 log.cc:824] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:45.278573 25297 log.cc:824] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:45.278564 25296 log.cc:824] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:45.280433 25296 tablet_bootstrap.cc:492] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: No bootstrap required, opened a new log
I20260807 22:46:45.280501 25299 tablet_bootstrap.cc:492] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: No bootstrap required, opened a new log
I20260807 22:46:45.280505 25297 tablet_bootstrap.cc:492] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: No bootstrap required, opened a new log
I20260807 22:46:45.280622 25296 ts_tablet_manager.cc:1397] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:46:45.280689 25299 ts_tablet_manager.cc:1397] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 22:46:45.280687 25297 ts_tablet_manager.cc:1397] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:46:45.283674 25296 raft_consensus.cc:348] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.283880 25296 raft_consensus.cc:374] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:45.283941 25296 raft_consensus.cc:729] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9827af2bab1a4e988724add08063fba6, State: Initialized, Role: FOLLOWER
I20260807 22:46:45.284345 25296 consensus_queue.cc:260] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.284376 25299 raft_consensus.cc:348] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.284376 25297 raft_consensus.cc:348] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.284613 25297 raft_consensus.cc:374] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:45.284613 25299 raft_consensus.cc:374] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:45.284662 25299 raft_consensus.cc:729] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf46c922aacd4adeb57cf0fc1ddc9b74, State: Initialized, Role: FOLLOWER
I20260807 22:46:45.284670 25297 raft_consensus.cc:729] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e71076886db40df89ab0940d9cee2a0, State: Initialized, Role: FOLLOWER
I20260807 22:46:45.285061 25297 consensus_queue.cc:260] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.285063 25299 consensus_queue.cc:260] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.285274 25296 ts_tablet_manager.cc:1428] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 22:46:45.285624 25093 heartbeater.cc:503] Master 127.26.189.254:39149 was elected leader, sending a full tablet report...
I20260807 22:46:45.285985 25297 ts_tablet_manager.cc:1428] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 22:46:45.286002 25270 heartbeater.cc:503] Master 127.26.189.254:39149 was elected leader, sending a full tablet report...
I20260807 22:46:45.286235 24901 heartbeater.cc:503] Master 127.26.189.254:39149 was elected leader, sending a full tablet report...
I20260807 22:46:45.287204 25299 ts_tablet_manager.cc:1428] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 22:46:45.294487 24903 tablet.cc:2369] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:45.333382 25308 raft_consensus.cc:482] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:45.333617 25308 raft_consensus.cc:504] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.335165 25308 leader_election.cc:288] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e71076886db40df89ab0940d9cee2a0 (127.26.189.195:36205), 9827af2bab1a4e988724add08063fba6 (127.26.189.194:35581)
I20260807 22:46:45.339272 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.335305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:44573 (local address 127.26.189.195:36205)
0807 22:46:45.335613 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.335617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.335715 (+ 98us) server_negotiation.cc:407] Connection header received
0807 22:46:45.335953 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.335956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.336007 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.336080 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:45.336374 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.336970 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.338286 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.338702 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.338795 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.338841 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.339070 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.339142 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260807 22:46:45.339326 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.335137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:35581 (local address 127.26.189.193:60547)
0807 22:46:45.335695 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.335708 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.335825 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.336075 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.336078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.336093 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:45.336217 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.336223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.337095 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.337099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.338189 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.338199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.338892 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.338896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.338970 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.338990 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.339056 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.339148 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":219,"spinlock_wait_cycles":2304,"thread_start_us":108,"threads_started":1}
I20260807 22:46:45.339306 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.335285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:60547 (local address 127.26.189.194:35581)
0807 22:46:45.335556 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.335562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.335794 (+ 232us) server_negotiation.cc:407] Connection header received
0807 22:46:45.335914 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.335918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.335974 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.336041 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:45.336331 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.336975 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.338339 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.338766 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.338896 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.338952 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.339089 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.339167 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 22:46:45.339357 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.335137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:36205 (local address 127.26.189.193:44573)
0807 22:46:45.335616 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.335634 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.335825 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.336106 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.336109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.336128 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:45.336268 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.336276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.337082 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.337086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.338168 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.338179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.338825 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.338830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.338929 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.338966 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.339052 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.339166 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":77,"spinlock_wait_cycles":13952,"thread_start_us":107,"threads_started":1}
I20260807 22:46:45.339927 25028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6320f15cdc4f45a8f386fe9721d328" candidate_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9827af2bab1a4e988724add08063fba6" is_pre_election: true
I20260807 22:46:45.340153 25212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6320f15cdc4f45a8f386fe9721d328" candidate_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e71076886db40df89ab0940d9cee2a0" is_pre_election: true
I20260807 22:46:45.340251 25028 raft_consensus.cc:2454] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf46c922aacd4adeb57cf0fc1ddc9b74 in term 0.
I20260807 22:46:45.340436 25212 raft_consensus.cc:2454] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf46c922aacd4adeb57cf0fc1ddc9b74 in term 0.
I20260807 22:46:45.340714 24735 leader_election.cc:302] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9827af2bab1a4e988724add08063fba6, cf46c922aacd4adeb57cf0fc1ddc9b74; no voters:
I20260807 22:46:45.341013 25308 raft_consensus.cc:2781] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:45.341099 25308 raft_consensus.cc:482] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:45.341137 25308 raft_consensus.cc:3037] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:45.342259 25308 raft_consensus.cc:504] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.342638 25308 leader_election.cc:288] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [CANDIDATE]: Term 1 election: Requested vote from peers 9e71076886db40df89ab0940d9cee2a0 (127.26.189.195:36205), 9827af2bab1a4e988724add08063fba6 (127.26.189.194:35581)
I20260807 22:46:45.343011 25028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6320f15cdc4f45a8f386fe9721d328" candidate_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9827af2bab1a4e988724add08063fba6"
I20260807 22:46:45.343140 25028 raft_consensus.cc:3037] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:45.344215 25028 raft_consensus.cc:2454] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf46c922aacd4adeb57cf0fc1ddc9b74 in term 1.
I20260807 22:46:45.344480 25212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6320f15cdc4f45a8f386fe9721d328" candidate_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e71076886db40df89ab0940d9cee2a0"
I20260807 22:46:45.344532 24735 leader_election.cc:302] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9827af2bab1a4e988724add08063fba6, cf46c922aacd4adeb57cf0fc1ddc9b74; no voters:
I20260807 22:46:45.344605 25212 raft_consensus.cc:3037] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:45.344703 25308 raft_consensus.cc:2781] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:45.344962 25308 raft_consensus.cc:686] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [term 1 LEADER]: Becoming Leader. State: Replica: cf46c922aacd4adeb57cf0fc1ddc9b74, State: Running, Role: LEADER
I20260807 22:46:45.345337 25308 consensus_queue.cc:237] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } }
I20260807 22:46:45.345994 25212 raft_consensus.cc:2454] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf46c922aacd4adeb57cf0fc1ddc9b74 in term 1.
I20260807 22:46:45.348798 24611 catalog_manager.cc:5495] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf46c922aacd4adeb57cf0fc1ddc9b74 (127.26.189.193). New cstate: current_term: 1 leader_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf46c922aacd4adeb57cf0fc1ddc9b74" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40557 } health_report { overall_health: HEALTHY } } }
I20260807 22:46:45.361562 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.358527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.26.189.193:40557)
0807 22:46:45.358629 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.358633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.358678 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:46:45.358804 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.358806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.358856 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.358938 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:45.359109 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.359597 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.360368 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.360581 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.360657 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.360866 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:45.361255 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:45.361340 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.361410 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.361459 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:46:45.367256 25212 raft_consensus.cc:1261] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 1 FOLLOWER]: Refusing update from remote peer cf46c922aacd4adeb57cf0fc1ddc9b74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:45.367312 25028 raft_consensus.cc:1261] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 1 FOLLOWER]: Refusing update from remote peer cf46c922aacd4adeb57cf0fc1ddc9b74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:45.367973 25312 consensus_queue.cc:1035] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e71076886db40df89ab0940d9cee2a0" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:45.368157 25308 consensus_queue.cc:1035] T 2a6320f15cdc4f45a8f386fe9721d328 P cf46c922aacd4adeb57cf0fc1ddc9b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9827af2bab1a4e988724add08063fba6" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 35581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:45.378055 25319 mvcc.cc:205] Tried to move back new op lower bound from 7316040930776854528 to 7316040930697830400. Current Snapshot: MvccSnapshot[applied={T|T < 7316040930776854528}]
I20260807 22:46:45.384064 24609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:45.384208 24609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:45.386502 24609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b15db500a7614487ae3de71eecfe1af7: Sending DeleteTablet for 3 replicas of tablet 2a6320f15cdc4f45a8f386fe9721d328
I20260807 22:46:45.387187 25187 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a6320f15cdc4f45a8f386fe9721d328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:45 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:57040
I20260807 22:46:45.387425 24804 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a6320f15cdc4f45a8f386fe9721d328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:45 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:49578
I20260807 22:46:45.387682 25000 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a6320f15cdc4f45a8f386fe9721d328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:45 UTC) from {username='kudu', principal='kudu/127.26.189.254@KRBTEST.COM'} at 127.0.0.1:35628
I20260807 22:46:45.387738 25330 tablet_replica.cc:330] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: stopping tablet replica
I20260807 22:46:45.388095 25333 tablet_replica.cc:330] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: stopping tablet replica
I20260807 22:46:45.388105 25330 raft_consensus.cc:2229] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:45.388419 25333 raft_consensus.cc:2229] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:45.388461 25330 raft_consensus.cc:2258] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:45.388677 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 24712
I20260807 22:46:45.390566 25330 ts_tablet_manager.cc:1905] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:45.391932 25333 raft_consensus.cc:2258] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:45.392743 25330 ts_tablet_manager.cc:1918] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:45.392830 25330 log.cc:1196] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/2a6320f15cdc4f45a8f386fe9721d328
I20260807 22:46:45.393103 25330 ts_tablet_manager.cc:1939] T 2a6320f15cdc4f45a8f386fe9721d328 P 9e71076886db40df89ab0940d9cee2a0: Deleting consensus metadata
I20260807 22:46:45.393379 25333 ts_tablet_manager.cc:1905] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:45.394109 24592 catalog_manager.cc:4841] TS 9e71076886db40df89ab0940d9cee2a0 (127.26.189.195:36205): tablet 2a6320f15cdc4f45a8f386fe9721d328 (table test-table [id=d495369534294a828600f54d90a469ec]) successfully deleted
I20260807 22:46:45.395910 25333 ts_tablet_manager.cc:1918] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:45.395987 25333 log.cc:1196] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/2a6320f15cdc4f45a8f386fe9721d328
I20260807 22:46:45.396301 25333 ts_tablet_manager.cc:1939] T 2a6320f15cdc4f45a8f386fe9721d328 P 9827af2bab1a4e988724add08063fba6: Deleting consensus metadata
I20260807 22:46:45.397334 24588 catalog_manager.cc:4841] TS 9827af2bab1a4e988724add08063fba6 (127.26.189.194:35581): tablet 2a6320f15cdc4f45a8f386fe9721d328 (table test-table [id=d495369534294a828600f54d90a469ec]) successfully deleted
W20260807 22:46:45.403376 24591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:40557 (error 108)
I20260807 22:46:45.403640 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 24909
I20260807 22:46:45.406625 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.406319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:40557 (local address 127.0.0.1:49610)
0807 22:46:45.406457 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.406548 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 22:46:45.406788 24591 catalog_manager.cc:4584] TS cf46c922aacd4adeb57cf0fc1ddc9b74 (127.26.189.193:40557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a6320f15cdc4f45a8f386fe9721d328: Network error: Client connection negotiation failed: client connection to 127.26.189.193:40557: connect: Connection refused (error 111)
I20260807 22:46:45.410470 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 25099
I20260807 22:46:45.419076 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 24561
2026-08-07T22:46:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2334 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T22:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:45Z Disabled control of system clock
I20260807 22:46:45.453039 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:42823
--webserver_interface=127.26.189.254
--webserver_port=0
--builtin_ntp_servers=127.26.189.212:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:42823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:46:45.561650 25339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:45.561889 25339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:45.561923 25339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:45.565868 25339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:45.565922 25339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:45.565936 25339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:45.565953 25339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:45.565966 25339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:45.570644 25339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:42823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:45.571602 25339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:45.572897 25339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:45.580822 25352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:45.580816 25350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:45.580969 25339 server_base.cc:1034] running on GCE node
W20260807 22:46:45.581147 25349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:45.581645 25339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:45.582705 25339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:45.583858 25339 hybrid_clock.cc:648] HybridClock initialized: now 1786142805583832 us; error 39 us; skew 500 ppm
I20260807 22:46:45.585747 25339 webserver.cc:460] Webserver started at http://127.26.189.254:39519/ using document root <none> and password file <none>
I20260807 22:46:45.586278 25339 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:45.586380 25339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:45.586580 25339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:45.588547 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "3d469177d3244867b555d35250ebea2a"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "7294da07f988d79881b1eb56f8966501"
server_key_iv: "9c54cf23a0118042ed24cdf36097d988"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.589080 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "3d469177d3244867b555d35250ebea2a"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "7294da07f988d79881b1eb56f8966501"
server_key_iv: "9c54cf23a0118042ed24cdf36097d988"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.592640 25339 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:46:45.594897 25363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.595994 25339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:45.596092 25339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "3d469177d3244867b555d35250ebea2a"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "7294da07f988d79881b1eb56f8966501"
server_key_iv: "9c54cf23a0118042ed24cdf36097d988"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.596200 25339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:45.628535 25339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:45.629199 25339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:45.629371 25339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:45.636152 25339 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:42823
I20260807 22:46:45.636173 25449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:42823 every 8 connection(s)
I20260807 22:46:45.637027 25339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:45.638592 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 25339
I20260807 22:46:45.638717 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:45.638994 27383 external_mini_cluster.cc:1428] Setting key 58bef02dd3a2fdb2ab9bc17cd2bc4f2b
I20260807 22:46:45.641031 25452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:45.646457 25452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a: Bootstrap starting.
I20260807 22:46:45.647599 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.640214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.26.189.254:42823)
0807 22:46:45.640802 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.640822 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.640855 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:46:45.641550 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.641570 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.641960 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.642262 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:45.642706 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.643684 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.644573 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.645024 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.645179 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.645973 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:45.645979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:45.645991 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:45.646409 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:45.646464 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:45.646484 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:45.646569 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.646707 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.646947 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":123,"threads_started":1}
I20260807 22:46:45.648638 25452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:45.649322 25452 log.cc:824] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:45.651079 25452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a: No bootstrap required, opened a new log
I20260807 22:46:45.654548 25452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d469177d3244867b555d35250ebea2a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42823 } }
I20260807 22:46:45.654757 25452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:45.654821 25452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d469177d3244867b555d35250ebea2a, State: Initialized, Role: FOLLOWER
I20260807 22:46:45.655196 25452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d469177d3244867b555d35250ebea2a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42823 } }
I20260807 22:46:45.655319 25452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:45.655380 25452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:45.655474 25452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:45.656291 25452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d469177d3244867b555d35250ebea2a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42823 } }
I20260807 22:46:45.656598 25452 leader_election.cc:302] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d469177d3244867b555d35250ebea2a; no voters:
I20260807 22:46:45.656893 25452 leader_election.cc:288] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:45.656953 25459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:45.657187 25459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [term 1 LEADER]: Becoming Leader. State: Replica: 3d469177d3244867b555d35250ebea2a, State: Running, Role: LEADER
I20260807 22:46:45.657459 25459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d469177d3244867b555d35250ebea2a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42823 } }
I20260807 22:46:45.657966 25452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:45.659040 25461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d469177d3244867b555d35250ebea2a. Latest consensus state: current_term: 1 leader_uuid: "3d469177d3244867b555d35250ebea2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d469177d3244867b555d35250ebea2a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42823 } } }
I20260807 22:46:45.659164 25461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:45.659416 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d469177d3244867b555d35250ebea2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d469177d3244867b555d35250ebea2a" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 42823 } } }
I20260807 22:46:45.659505 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:45.659613 25475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:45.662864 25475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:45.668455 25475 catalog_manager.cc:1347] Generated new cluster ID: cec04eb022c44d59abcdfb062be79337
I20260807 22:46:45.668514 25475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:45.695276 25475 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:45.696535 25475 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:45.709263 25475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a: Generated new TSK 0
I20260807 22:46:45.710047 25475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:45.722409 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.193:0
--local_ip_for_outbound_sockets=127.26.189.193
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42823
--builtin_ntp_servers=127.26.189.212:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:46:45.831044 25492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:45.831275 25492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:45.831310 25492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:45.835158 25492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:45.835202 25492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:45.835254 25492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.193
I20260807 22:46:45.839771 25492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.189.193
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.193
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:45.840725 25492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:45.842127 25492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:45.851260 25502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:45.851250 25500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:45.851251 25499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:45.851500 25492 server_base.cc:1034] running on GCE node
I20260807 22:46:45.851962 25492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:45.852600 25492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:45.853780 25492 hybrid_clock.cc:648] HybridClock initialized: now 1786142805853757 us; error 41 us; skew 500 ppm
I20260807 22:46:45.855664 25492 webserver.cc:460] Webserver started at http://127.26.189.193:32845/ using document root <none> and password file <none>
I20260807 22:46:45.856195 25492 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:45.856261 25492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:45.856439 25492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:45.858481 25492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data/instance:
uuid: "063f350fffad48f4b66f668df140acc0"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "9fcbd74725f9aaadb7e157b96eede914"
server_key_iv: "407ae6b27e0fe1d6524e73c0682cb9a3"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.859090 25492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance:
uuid: "063f350fffad48f4b66f668df140acc0"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "9fcbd74725f9aaadb7e157b96eede914"
server_key_iv: "407ae6b27e0fe1d6524e73c0682cb9a3"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.862680 25492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:46:45.864925 25511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.866060 25492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:45.866163 25492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal
uuid: "063f350fffad48f4b66f668df140acc0"
format_stamp: "Formatted at 2026-08-07 22:46:45 on dist-test-slave-4tdj"
server_key: "9fcbd74725f9aaadb7e157b96eede914"
server_key_iv: "407ae6b27e0fe1d6524e73c0682cb9a3"
server_key_version: "encryptionkey@0"
I20260807 22:46:45.866248 25492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:45.877372 25492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:45.877954 25492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:45.878121 25492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:45.878677 25492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:45.879555 25492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:45.879604 25492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.879652 25492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:45.879683 25492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:45.887949 25492 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.193:40875
I20260807 22:46:45.887979 25669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.193:40875 every 8 connection(s)
I20260807 22:46:45.888758 25492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/data/info.pb
I20260807 22:46:45.889326 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 25492
I20260807 22:46:45.889454 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-0/wal/instance
I20260807 22:46:45.889787 27383 external_mini_cluster.cc:1428] Setting key b5e1fd6d0fd380879dcb7d9344c7c33e
I20260807 22:46:45.891916 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.194:0
--local_ip_for_outbound_sockets=127.26.189.194
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42823
--builtin_ntp_servers=127.26.189.212:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:46:45.898247 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.890561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.193:35873 (local address 127.26.189.254:42823)
0807 22:46:45.890697 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:45.890702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:45.891778 (+ 1076us) server_negotiation.cc:407] Connection header received
0807 22:46:45.893104 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:45.893107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:45.893160 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:45.893231 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:45.894370 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.894956 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.896138 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.896498 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.896614 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.897751 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:45.897756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:45.897759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:45.897774 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:45.897798 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:45.897808 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:45.897889 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:45.898004 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 22:46:45.898098 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:46:45.899016 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:45.890980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42823 (local address 127.26.189.193:35873)
0807 22:46:45.891646 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:45.891685 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:45.892905 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:45.893423 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:45.893433 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:45.893550 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:45.894191 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.894209 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.895093 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.895096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.895996 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:45.896007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:45.896650 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:45.896654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:45.896829 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:45.897464 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:45.897466 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:45.897503 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:45.897597 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:45.897885 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:45.897889 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:45.897893 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:46:45.897976 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:46:45.898179 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":121,"threads_started":1}
I20260807 22:46:45.900168 25671 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42823
I20260807 22:46:45.900403 25671 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:45.900878 25671 heartbeater.cc:511] Master 127.26.189.254:42823 requested a full tablet report, sending...
I20260807 22:46:45.902522 25393 ts_manager.cc:194] Registered new tserver with Master: 063f350fffad48f4b66f668df140acc0 (127.26.189.193:40875)
I20260807 22:46:45.904150 25393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.193:35873
W20260807 22:46:46.001820 25678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:46.002070 25678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:46.002112 25678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:46.006100 25678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:46.006155 25678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:46.006206 25678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.194
I20260807 22:46:46.010840 25678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.189.194
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.194
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:46.011835 25678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:46.013125 25678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:46.022202 25684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:46.022202 25688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:46.022308 25685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:46.022521 25678 server_base.cc:1034] running on GCE node
I20260807 22:46:46.022874 25678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:46.023447 25678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:46.024629 25678 hybrid_clock.cc:648] HybridClock initialized: now 1786142806024607 us; error 27 us; skew 500 ppm
I20260807 22:46:46.026420 25678 webserver.cc:460] Webserver started at http://127.26.189.194:36189/ using document root <none> and password file <none>
I20260807 22:46:46.026944 25678 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:46.027012 25678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:46.027237 25678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:46.029174 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data/instance:
uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854"
format_stamp: "Formatted at 2026-08-07 22:46:46 on dist-test-slave-4tdj"
server_key: "e5a394c27c272c3d85bae5f1c6f47487"
server_key_iv: "10d6ffc703551bc36be2d140f0443848"
server_key_version: "encryptionkey@0"
I20260807 22:46:46.029713 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance:
uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854"
format_stamp: "Formatted at 2026-08-07 22:46:46 on dist-test-slave-4tdj"
server_key: "e5a394c27c272c3d85bae5f1c6f47487"
server_key_iv: "10d6ffc703551bc36be2d140f0443848"
server_key_version: "encryptionkey@0"
I20260807 22:46:46.033161 25678 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 22:46:46.035517 25695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:46.036684 25678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:46:46.036795 25678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal
uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854"
format_stamp: "Formatted at 2026-08-07 22:46:46 on dist-test-slave-4tdj"
server_key: "e5a394c27c272c3d85bae5f1c6f47487"
server_key_iv: "10d6ffc703551bc36be2d140f0443848"
server_key_version: "encryptionkey@0"
I20260807 22:46:46.036887 25678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:46.047315 25678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:46.047890 25678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:46.048040 25678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:46.048564 25678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:46.049427 25678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:46.049472 25678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:46.049523 25678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:46.049556 25678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:46.057711 25678 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.194:46249
I20260807 22:46:46.057760 25867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.194:46249 every 8 connection(s)
I20260807 22:46:46.058629 25678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/data/info.pb
I20260807 22:46:46.058943 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 25678
I20260807 22:46:46.059070 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal/instance
I20260807 22:46:46.059430 27383 external_mini_cluster.cc:1428] Setting key cf89bee8560d0617af90cfdbecde5ead
I20260807 22:46:46.061532 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.189.195:0
--local_ip_for_outbound_sockets=127.26.189.195
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42823
--builtin_ntp_servers=127.26.189.212:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:46:46.068723 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.060439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:56331 (local address 127.26.189.254:42823)
0807 22:46:46.060564 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.060567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.061863 (+ 1296us) server_negotiation.cc:407] Connection header received
0807 22:46:46.063311 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.063313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.063360 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.063431 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:46.064535 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.065041 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.066698 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.066921 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.067016 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.068254 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:46.068257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:46.068259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:46.068272 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:46.068291 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:46.068299 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:46.068356 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.068550 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.068609 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:46:46.069864 25873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.060996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42823 (local address 127.26.189.194:56331)
0807 22:46:46.061709 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.061760 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.063061 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.063609 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.063618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.063725 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:46.064387 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.064407 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.065176 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.065180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.066552 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.066565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.067084 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.067088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.067300 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.067934 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:46.067937 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:46.067993 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:46.068089 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:46.068377 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:46.068386 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:46.068392 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.068525 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.068824 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260807 22:46:46.071187 25868 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42823
I20260807 22:46:46.071409 25868 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:46.071903 25868 heartbeater.cc:511] Master 127.26.189.254:42823 requested a full tablet report, sending...
I20260807 22:46:46.072894 25393 ts_manager.cc:194] Registered new tserver with Master: 8bd39004ba5e4f7bbfd1e09a32d9e854 (127.26.189.194:46249)
I20260807 22:46:46.073436 25393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.194:56331
W20260807 22:46:46.172308 25875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:46.172570 25875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:46.172626 25875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:46.176558 25875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:46.176606 25875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:46.176685 25875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.189.195
I20260807 22:46:46.181298 25875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.189.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.189.195
--webserver_port=0
--tserver_master_addrs=127.26.189.254:42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.189.195
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:46.182271 25875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:46.183593 25875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:46.192826 25880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:46.192914 25875 server_base.cc:1034] running on GCE node
W20260807 22:46:46.192836 25884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:46.192848 25881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:46.193472 25875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:46.194108 25875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:46.195271 25875 hybrid_clock.cc:648] HybridClock initialized: now 1786142806195245 us; error 36 us; skew 500 ppm
I20260807 22:46:46.196991 25875 webserver.cc:460] Webserver started at http://127.26.189.195:39223/ using document root <none> and password file <none>
I20260807 22:46:46.197517 25875 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:46.197590 25875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:46.197784 25875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:46.199790 25875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data/instance:
uuid: "45050ece93cf40948aef77e91dc85668"
format_stamp: "Formatted at 2026-08-07 22:46:46 on dist-test-slave-4tdj"
server_key: "739998d19c79d5cb5153538de4d16aab"
server_key_iv: "af8092fe93423ccfc1c695ca0b9edf4a"
server_key_version: "encryptionkey@0"
I20260807 22:46:46.200300 25875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance:
uuid: "45050ece93cf40948aef77e91dc85668"
format_stamp: "Formatted at 2026-08-07 22:46:46 on dist-test-slave-4tdj"
server_key: "739998d19c79d5cb5153538de4d16aab"
server_key_iv: "af8092fe93423ccfc1c695ca0b9edf4a"
server_key_version: "encryptionkey@0"
I20260807 22:46:46.203748 25875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:46:46.205945 25897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:46.207151 25875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:46:46.207257 25875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal
uuid: "45050ece93cf40948aef77e91dc85668"
format_stamp: "Formatted at 2026-08-07 22:46:46 on dist-test-slave-4tdj"
server_key: "739998d19c79d5cb5153538de4d16aab"
server_key_iv: "af8092fe93423ccfc1c695ca0b9edf4a"
server_key_version: "encryptionkey@0"
I20260807 22:46:46.207345 25875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:46.218168 25875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:46.218813 25875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:46.218967 25875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:46.219483 25875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:46:46.220356 25875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:46:46.220404 25875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:46.220454 25875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:46:46.220479 25875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:46.229063 25875 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.195:33027
I20260807 22:46:46.229097 26054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.195:33027 every 8 connection(s)
I20260807 22:46:46.229892 25875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/data/info.pb
I20260807 22:46:46.238487 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.231589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.195:41111 (local address 127.26.189.254:42823)
0807 22:46:46.231728 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.231732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.232729 (+ 997us) server_negotiation.cc:407] Connection header received
0807 22:46:46.234117 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.234120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.234178 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.234248 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:46.235115 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.235631 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.236737 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.236948 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.237045 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.238016 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:46.238019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:46.238021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:46.238032 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:46.238057 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:46.238064 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:46.238122 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.238256 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.238344 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:46:46.238819 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 25875
I20260807 22:46:46.238919 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal/instance
I20260807 22:46:46.239181 27383 external_mini_cluster.cc:1428] Setting key 59b3b2fbb653ffe17b7979a7cefb4081
I20260807 22:46:46.239326 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.231993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:42823 (local address 127.26.189.195:41111)
0807 22:46:46.232597 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.232634 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.233910 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.234441 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.234447 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.234529 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:46:46.234979 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.234991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.235751 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.235754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.236622 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.236631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.237086 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.237090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.237243 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.237780 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:46:46.237782 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:46:46.237818 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:46:46.237877 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:46:46.238133 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:46:46.238142 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:46:46.238148 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.238248 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.238524 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":114,"threads_started":1}
I20260807 22:46:46.240401 26055 heartbeater.cc:347] Connected to a master server at 127.26.189.254:42823
I20260807 22:46:46.240628 26055 heartbeater.cc:464] Registering TS with master...
I20260807 22:46:46.241158 26055 heartbeater.cc:511] Master 127.26.189.254:42823 requested a full tablet report, sending...
I20260807 22:46:46.242077 25392 ts_manager.cc:194] Registered new tserver with Master: 45050ece93cf40948aef77e91dc85668 (127.26.189.195:33027)
I20260807 22:46:46.242735 25392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.189.195:41111
I20260807 22:46:46.252960 27383 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:46:46.258541 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.254899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.26.189.254:42823)
0807 22:46:46.255069 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.255074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.255207 (+ 133us) server_negotiation.cc:407] Connection header received
0807 22:46:46.255302 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.255304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.255346 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.255414 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:46.255707 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.256254 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.257040 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.257242 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.257312 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.258109 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:46.258113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:46.258116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:46.258130 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:46.258158 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:46.258167 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:46.258227 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.258358 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.258409 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 22:46:46.261385 25392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:46:46.264335 25392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:46:46.278034 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.272255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33027 (local address 127.0.0.1:41450)
0807 22:46:46.272734 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.272783 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.272904 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.273785 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.273791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.273815 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:46.273948 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.273955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.275382 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.275385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.276601 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.276610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.277526 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.277530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.277632 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.277675 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.277776 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.277890 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":4,"thread_start_us":68,"threads_started":1}
I20260807 22:46:46.278277 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.273041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.26.189.194:46249)
0807 22:46:46.273596 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.273612 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.273920 (+ 308us) server_negotiation.cc:407] Connection header received
0807 22:46:46.274000 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.274006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.274195 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.274370 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:46.274542 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.275556 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.276960 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.277559 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.277668 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.277825 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.278025 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.278126 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":116,"threads_started":1}
I20260807 22:46:46.278431 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.272427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.26.189.195:33027)
0807 22:46:46.273294 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.273306 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.273331 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:46:46.273396 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.273400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.273627 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.273775 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:46.274086 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.275264 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.276736 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.277401 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.277516 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.277684 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.278135 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.278234 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":150,"threads_started":1}
I20260807 22:46:46.278678 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.272369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.194:46249 (local address 127.0.0.1:52446)
0807 22:46:46.273729 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.273741 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.273832 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.274329 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.274331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.274350 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:46.274447 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.274453 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.275656 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.275660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.276826 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.276838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.277768 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.277772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.277863 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.277886 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.277961 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.278029 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1301,"mutex_wait_us":19,"spinlock_wait_cycles":2944,"thread_start_us":124,"threads_started":1}
I20260807 22:46:46.278859 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.272906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:40875 (local address 127.0.0.1:49062)
0807 22:46:46.273993 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.274006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.274101 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.274713 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.274715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.274730 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:46.274810 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.274814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.275913 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.275916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.277047 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.277056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.278456 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.278460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.278571 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.278597 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.278664 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.278732 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":26,"thread_start_us":95,"threads_started":1}
I20260807 22:46:46.279091 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.273051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.26.189.193:40875)
0807 22:46:46.273385 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.273396 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.274195 (+ 799us) server_negotiation.cc:407] Connection header received
0807 22:46:46.274264 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.274267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.274466 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.274598 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:46.274915 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.275800 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.277163 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.277888 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.278001 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.278407 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.278713 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.278883 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":110,"threads_started":1}
I20260807 22:46:46.281277 25962 tablet_service.cc:1467] Processing CreateTablet for tablet b917c3748ef3452295e9bcafb5fa70c9 (DEFAULT_TABLE table=test-table [id=248c47d7f5984d3599cc8c3cd60d03e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:46.281435 25765 tablet_service.cc:1467] Processing CreateTablet for tablet b917c3748ef3452295e9bcafb5fa70c9 (DEFAULT_TABLE table=test-table [id=248c47d7f5984d3599cc8c3cd60d03e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:46.281800 25579 tablet_service.cc:1467] Processing CreateTablet for tablet b917c3748ef3452295e9bcafb5fa70c9 (DEFAULT_TABLE table=test-table [id=248c47d7f5984d3599cc8c3cd60d03e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:46:46.282243 25962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b917c3748ef3452295e9bcafb5fa70c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:46.282523 25765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b917c3748ef3452295e9bcafb5fa70c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:46.282759 25579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b917c3748ef3452295e9bcafb5fa70c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:46.288066 26081 tablet_bootstrap.cc:492] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Bootstrap starting.
I20260807 22:46:46.289810 26082 tablet_bootstrap.cc:492] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: Bootstrap starting.
I20260807 22:46:46.290634 26083 tablet_bootstrap.cc:492] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Bootstrap starting.
I20260807 22:46:46.290827 26081 tablet_bootstrap.cc:654] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:46.291514 26081 log.cc:824] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:46.291837 26082 tablet_bootstrap.cc:654] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:46.292579 26083 tablet_bootstrap.cc:654] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:46.292646 26082 log.cc:824] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:46.293283 26081 tablet_bootstrap.cc:492] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: No bootstrap required, opened a new log
I20260807 22:46:46.293322 26083 log.cc:824] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:46.293481 26081 ts_tablet_manager.cc:1397] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 22:46:46.294687 26082 tablet_bootstrap.cc:492] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: No bootstrap required, opened a new log
I20260807 22:46:46.294860 26082 ts_tablet_manager.cc:1397] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:46:46.295224 26083 tablet_bootstrap.cc:492] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: No bootstrap required, opened a new log
I20260807 22:46:46.295401 26083 ts_tablet_manager.cc:1397] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 22:46:46.297690 26081 raft_consensus.cc:348] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.297787 26082 raft_consensus.cc:348] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.297948 26081 raft_consensus.cc:374] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:46.297979 26082 raft_consensus.cc:374] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:46.298007 26081 raft_consensus.cc:729] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45050ece93cf40948aef77e91dc85668, State: Initialized, Role: FOLLOWER
I20260807 22:46:46.298029 26082 raft_consensus.cc:729] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 063f350fffad48f4b66f668df140acc0, State: Initialized, Role: FOLLOWER
I20260807 22:46:46.298456 26082 consensus_queue.cc:260] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.298467 26081 consensus_queue.cc:260] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.299142 25671 heartbeater.cc:503] Master 127.26.189.254:42823 was elected leader, sending a full tablet report...
I20260807 22:46:46.299374 26082 ts_tablet_manager.cc:1428] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 22:46:46.299397 26081 ts_tablet_manager.cc:1428] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 22:46:46.299695 26055 heartbeater.cc:503] Master 127.26.189.254:42823 was elected leader, sending a full tablet report...
I20260807 22:46:46.299976 26083 raft_consensus.cc:348] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.300271 26083 raft_consensus.cc:374] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:46.300328 26083 raft_consensus.cc:729] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bd39004ba5e4f7bbfd1e09a32d9e854, State: Initialized, Role: FOLLOWER
I20260807 22:46:46.300858 26083 consensus_queue.cc:260] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.301468 25868 heartbeater.cc:503] Master 127.26.189.254:42823 was elected leader, sending a full tablet report...
I20260807 22:46:46.302800 26083 ts_tablet_manager.cc:1428] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 22:46:46.309629 25870 tablet.cc:2369] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:46:46.389815 25674 tablet.cc:2369] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:46:46.409349 26093 raft_consensus.cc:482] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:46:46.409569 26093 raft_consensus.cc:504] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.410813 26093 leader_election.cc:288] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45050ece93cf40948aef77e91dc85668 (127.26.189.195:33027), 063f350fffad48f4b66f668df140acc0 (127.26.189.193:40875)
I20260807 22:46:46.414654 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.411192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:34183 (local address 127.26.189.195:33027)
0807 22:46:46.411339 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.411344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.411363 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:46:46.411429 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.411432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.411482 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.411585 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:46.411889 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.412476 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.413658 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.414064 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.414164 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.414216 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.414451 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.414530 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:46:46.414655 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.411330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.189.194:41393 (local address 127.26.189.193:40875)
0807 22:46:46.411460 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.411463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.411636 (+ 173us) server_negotiation.cc:407] Connection header received
0807 22:46:46.411733 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.411737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.411810 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.411881 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:46:46.412168 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.412690 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.413702 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.414086 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.414194 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.414240 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.414482 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.414553 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:46:46.414657 25873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.411069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.195:33027 (local address 127.26.189.194:34183)
0807 22:46:46.411216 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.411232 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.411331 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.411600 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.411603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.411626 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:46.411767 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.411775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.412606 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.412610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.413497 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.413541 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.414179 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.414183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.414278 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.414358 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.414420 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.414494 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":43,"spinlock_wait_cycles":768}
I20260807 22:46:46.414671 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.411093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:40875 (local address 127.26.189.194:41393)
0807 22:46:46.411545 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.411559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:46:46.411651 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:46:46.411901 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:46:46.411903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:46:46.411921 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:46:46.412056 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.412063 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.412797 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.412799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.413575 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:46:46.413584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.414204 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.414207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:46:46.414361 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.414379 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 22:46:46.414453 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:46:46.414522 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":21,"thread_start_us":125,"threads_started":1}
I20260807 22:46:46.415333 25606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b917c3748ef3452295e9bcafb5fa70c9" candidate_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "063f350fffad48f4b66f668df140acc0" is_pre_election: true
I20260807 22:46:46.415362 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b917c3748ef3452295e9bcafb5fa70c9" candidate_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45050ece93cf40948aef77e91dc85668" is_pre_election: true
I20260807 22:46:46.415621 25990 raft_consensus.cc:2454] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bd39004ba5e4f7bbfd1e09a32d9e854 in term 0.
I20260807 22:46:46.415627 25606 raft_consensus.cc:2454] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bd39004ba5e4f7bbfd1e09a32d9e854 in term 0.
I20260807 22:46:46.416072 25700 leader_election.cc:302] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45050ece93cf40948aef77e91dc85668, 8bd39004ba5e4f7bbfd1e09a32d9e854; no voters:
I20260807 22:46:46.416334 26093 raft_consensus.cc:2781] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:46:46.416421 26093 raft_consensus.cc:482] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:46:46.416469 26093 raft_consensus.cc:3037] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:46.417476 26093 raft_consensus.cc:504] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.417784 26093 leader_election.cc:288] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [CANDIDATE]: Term 1 election: Requested vote from peers 45050ece93cf40948aef77e91dc85668 (127.26.189.195:33027), 063f350fffad48f4b66f668df140acc0 (127.26.189.193:40875)
I20260807 22:46:46.418099 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b917c3748ef3452295e9bcafb5fa70c9" candidate_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45050ece93cf40948aef77e91dc85668"
I20260807 22:46:46.418123 25606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b917c3748ef3452295e9bcafb5fa70c9" candidate_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "063f350fffad48f4b66f668df140acc0"
I20260807 22:46:46.418246 25606 raft_consensus.cc:3037] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:46.418246 25990 raft_consensus.cc:3037] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:46.419595 25606 raft_consensus.cc:2454] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bd39004ba5e4f7bbfd1e09a32d9e854 in term 1.
I20260807 22:46:46.419595 25990 raft_consensus.cc:2454] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bd39004ba5e4f7bbfd1e09a32d9e854 in term 1.
I20260807 22:46:46.419909 25700 leader_election.cc:302] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45050ece93cf40948aef77e91dc85668, 8bd39004ba5e4f7bbfd1e09a32d9e854; no voters:
I20260807 22:46:46.420156 26093 raft_consensus.cc:2781] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:46.420400 26093 raft_consensus.cc:686] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 1 LEADER]: Becoming Leader. State: Replica: 8bd39004ba5e4f7bbfd1e09a32d9e854, State: Running, Role: LEADER
I20260807 22:46:46.420930 26093 consensus_queue.cc:237] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } }
I20260807 22:46:46.424139 25392 catalog_manager.cc:5495] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bd39004ba5e4f7bbfd1e09a32d9e854 (127.26.189.194). New cstate: current_term: 1 leader_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bd39004ba5e4f7bbfd1e09a32d9e854" member_type: VOTER last_known_addr { host: "127.26.189.194" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 } health_report { overall_health: UNKNOWN } } }
I20260807 22:46:46.465636 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.462147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.26.189.194:46249)
0807 22:46:46.462282 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:46.462287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:46.462328 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:46:46.462472 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:46.462474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:46.462541 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:46.462636 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:46:46.462881 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.463390 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.464223 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:46.464438 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:46.464526 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:46.464826 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:46:46.465257 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:46:46.465367 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:46.465442 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 22:46:46.465498 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:46:46.471686 25606 raft_consensus.cc:1261] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 1 FOLLOWER]: Refusing update from remote peer 8bd39004ba5e4f7bbfd1e09a32d9e854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:46.471685 25990 raft_consensus.cc:1261] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 1 FOLLOWER]: Refusing update from remote peer 8bd39004ba5e4f7bbfd1e09a32d9e854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:46:46.472465 26095 consensus_queue.cc:1035] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45050ece93cf40948aef77e91dc85668" member_type: VOTER last_known_addr { host: "127.26.189.195" port: 33027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:46.472663 26093 consensus_queue.cc:1035] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "063f350fffad48f4b66f668df140acc0" member_type: VOTER last_known_addr { host: "127.26.189.193" port: 40875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:46:46.479702 26098 mvcc.cc:205] Tried to move back new op lower bound from 7316040935300100096 to 7316040935102713856. Current Snapshot: MvccSnapshot[applied={T|T < 7316040935300100096}]
I20260807 22:46:46.488329 25393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:46.488467 25393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:46:46.490725 25393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d469177d3244867b555d35250ebea2a: Sending DeleteTablet for 3 replicas of tablet b917c3748ef3452295e9bcafb5fa70c9
I20260807 22:46:46.491372 25962 tablet_service.cc:1514] Processing DeleteTablet for tablet b917c3748ef3452295e9bcafb5fa70c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:46 UTC) from {username='slave'} at 127.0.0.1:41450
I20260807 22:46:46.491429 25765 tablet_service.cc:1514] Processing DeleteTablet for tablet b917c3748ef3452295e9bcafb5fa70c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:46 UTC) from {username='slave'} at 127.0.0.1:52446
I20260807 22:46:46.491819 26116 tablet_replica.cc:330] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: stopping tablet replica
I20260807 22:46:46.491819 25579 tablet_service.cc:1514] Processing DeleteTablet for tablet b917c3748ef3452295e9bcafb5fa70c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:46:46 UTC) from {username='slave'} at 127.0.0.1:49062
I20260807 22:46:46.492100 26116 raft_consensus.cc:2229] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:46.492244 26119 tablet_replica.cc:330] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0: stopping tablet replica
I20260807 22:46:46.492378 26116 raft_consensus.cc:2258] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:46.492509 26119 raft_consensus.cc:2229] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:46:46.492916 26119 raft_consensus.cc:2258] T b917c3748ef3452295e9bcafb5fa70c9 P 063f350fffad48f4b66f668df140acc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:46.493024 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 25492
I20260807 22:46:46.493767 26116 ts_tablet_manager.cc:1905] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:46.496634 26116 ts_tablet_manager.cc:1918] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:46.496706 26116 log.cc:1196] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-2/wal/wals/b917c3748ef3452295e9bcafb5fa70c9
I20260807 22:46:46.496927 26116 ts_tablet_manager.cc:1939] T b917c3748ef3452295e9bcafb5fa70c9 P 45050ece93cf40948aef77e91dc85668: Deleting consensus metadata
I20260807 22:46:46.497889 25374 catalog_manager.cc:4841] TS 45050ece93cf40948aef77e91dc85668 (127.26.189.195:33027): tablet b917c3748ef3452295e9bcafb5fa70c9 (table test-table [id=248c47d7f5984d3599cc8c3cd60d03e0]) successfully deleted
I20260807 22:46:46.499907 26117 tablet_replica.cc:330] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: stopping tablet replica
I20260807 22:46:46.500196 26117 raft_consensus.cc:2229] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:46:46.500495 26117 raft_consensus.cc:2258] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:46:46.503644 26117 ts_tablet_manager.cc:1905] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:46:46.505954 26117 ts_tablet_manager.cc:1918] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:46:46.506044 26117 log.cc:1196] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Deleting WAL directory at /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142556986508-27383-0/minicluster-data/ts-1/wal/wals/b917c3748ef3452295e9bcafb5fa70c9
I20260807 22:46:46.506337 26117 ts_tablet_manager.cc:1939] T b917c3748ef3452295e9bcafb5fa70c9 P 8bd39004ba5e4f7bbfd1e09a32d9e854: Deleting consensus metadata
I20260807 22:46:46.507265 25372 catalog_manager.cc:4841] TS 8bd39004ba5e4f7bbfd1e09a32d9e854 (127.26.189.194:46249): tablet b917c3748ef3452295e9bcafb5fa70c9 (table test-table [id=248c47d7f5984d3599cc8c3cd60d03e0]) successfully deleted
W20260807 22:46:46.509121 25371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.189.193:40875 (error 108)
I20260807 22:46:46.509374 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 25678
I20260807 22:46:46.513506 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:46.513227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.193:40875 (local address 127.0.0.1:49078)
0807 22:46:46.513365 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:46:46.513433 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.193:40875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:46:46.513667 25371 catalog_manager.cc:4584] TS 063f350fffad48f4b66f668df140acc0 (127.26.189.193:40875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b917c3748ef3452295e9bcafb5fa70c9: Network error: Client connection negotiation failed: client connection to 127.26.189.193:40875: connect: Connection refused (error 111)
I20260807 22:46:46.517598 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 25875
I20260807 22:46:46.527201 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 25339
2026-08-07T22:46:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1107 ms)
[----------] 6 tests from EncryptionPolicyTest (10697 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:46 dist-test-slave-4tdj krb5kdc[26125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:46 dist-test-slave-4tdj krb5kdc[26125](info): set up 2 sockets
Aug 07 22:46:46 dist-test-slave-4tdj krb5kdc[26125](info): commencing operation
krb5kdc: starting...
W20260807 22:46:47.661659 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:47 dist-test-slave-4tdj krb5kdc[26125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:47.779354 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:47.886636 26142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:47.886878 26142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:47.886919 26142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:47.890767 26142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:47.890811 26142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:47.890828 26142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:47.890842 26142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:47.890856 26142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:47.895851 26142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:47.896781 26142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:47.897694 26142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:47.905810 26151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:47.905864 26155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:47.905963 26152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:47.906100 26142 server_base.cc:1034] running on GCE node
I20260807 22:46:47.906775 26142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:47.907864 26142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:47.909026 26142 hybrid_clock.cc:648] HybridClock initialized: now 1786142807909010 us; error 31 us; skew 500 ppm
Aug 07 22:46:47 dist-test-slave-4tdj krb5kdc[26125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142807, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:47.916700 26142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:47.917618 26142 webserver.cc:460] Webserver started at http://127.0.0.1:37225/ using document root <none> and password file <none>
I20260807 22:46:47.918165 26142 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:47.918231 26142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:47.918468 26142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:47.920506 26142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "1f41febb6fb54a4993995e3e269d19d9"
format_stamp: "Formatted at 2026-08-07 22:46:47 on dist-test-slave-4tdj"
server_key: "e0b11c6b2dc9b4e30d491cfd7bf98aaf"
server_key_iv: "a3cafb0e617fc0b842676c939e24f010"
server_key_version: "encryptionkey@0"
I20260807 22:46:47.921065 26142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "1f41febb6fb54a4993995e3e269d19d9"
format_stamp: "Formatted at 2026-08-07 22:46:47 on dist-test-slave-4tdj"
server_key: "e0b11c6b2dc9b4e30d491cfd7bf98aaf"
server_key_iv: "a3cafb0e617fc0b842676c939e24f010"
server_key_version: "encryptionkey@0"
I20260807 22:46:47.924831 26142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 22:46:47.927223 26164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:47.928467 26142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:47.928624 26142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "1f41febb6fb54a4993995e3e269d19d9"
format_stamp: "Formatted at 2026-08-07 22:46:47 on dist-test-slave-4tdj"
server_key: "e0b11c6b2dc9b4e30d491cfd7bf98aaf"
server_key_iv: "a3cafb0e617fc0b842676c939e24f010"
server_key_version: "encryptionkey@0"
I20260807 22:46:47.928719 26142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:47.957652 26142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:47.961030 26142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:47.961220 26142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:47.968427 26142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40381
I20260807 22:46:47.968467 26238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40381 every 8 connection(s)
I20260807 22:46:47.969379 26142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:47.972196 26239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:47.975296 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 26142
I20260807 22:46:47.975435 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:47.975692 27383 external_mini_cluster.cc:1428] Setting key ca9b364107e39ec9276336d751d3a085
I20260807 22:46:47.977960 26239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9: Bootstrap starting.
I20260807 22:46:47.980566 26239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:47.981407 26239 log.cc:824] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:47.983439 26239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9: No bootstrap required, opened a new log
Aug 07 22:46:47 dist-test-slave-4tdj krb5kdc[26125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:46:47.987527 26239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f41febb6fb54a4993995e3e269d19d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40381 } }
I20260807 22:46:47.987782 26239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:47.987850 26239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f41febb6fb54a4993995e3e269d19d9, State: Initialized, Role: FOLLOWER
I20260807 22:46:47.988349 26239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f41febb6fb54a4993995e3e269d19d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40381 } }
I20260807 22:46:47.988515 26239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:47.988586 26239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:47.988684 26239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:47.989758 26239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f41febb6fb54a4993995e3e269d19d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40381 } }
I20260807 22:46:47.990120 26239 leader_election.cc:302] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f41febb6fb54a4993995e3e269d19d9; no voters:
I20260807 22:46:47.990453 26239 leader_election.cc:288] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:47.990574 26247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:47.990717 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:47.976957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43151 (local address 127.0.0.1:40381)
0807 22:46:47.977388 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:47.977406 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:47.977446 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:46:47.978258 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:47.978281 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:47.978729 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:47.979086 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:47.979752 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:47.981218 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:47.982075 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:47.982762 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:47.982923 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:47.985459 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:47.985466 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:47.985481 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:47.985513 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:47.987446 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:47.987931 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:47.987937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:47.987943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:47.988083 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:47.988420 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:47.988423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:47.988424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:47.988948 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:47.989145 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:47.989470 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:46:47.989703 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":114,"threads_started":1}
I20260807 22:46:47.990803 26247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [term 1 LEADER]: Becoming Leader. State: Replica: 1f41febb6fb54a4993995e3e269d19d9, State: Running, Role: LEADER
I20260807 22:46:47.991282 26247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f41febb6fb54a4993995e3e269d19d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40381 } }
I20260807 22:46:47.991683 26239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:47.992825 26252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f41febb6fb54a4993995e3e269d19d9. Latest consensus state: current_term: 1 leader_uuid: "1f41febb6fb54a4993995e3e269d19d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f41febb6fb54a4993995e3e269d19d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40381 } } }
I20260807 22:46:47.993032 26252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:47.993517 26250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f41febb6fb54a4993995e3e269d19d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f41febb6fb54a4993995e3e269d19d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40381 } } }
I20260807 22:46:47.993522 26265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:47.993619 26250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:47.994879 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:47.996762 26265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:48.002636 26265 catalog_manager.cc:1347] Generated new cluster ID: 310d41b82e194efaa6c23d7c42350fc1
I20260807 22:46:48.002691 26265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:48.009318 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:47.996955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52675 (local address 127.0.0.1:40381)
0807 22:46:47.997056 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:47.997059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:47.997384 (+ 325us) server_negotiation.cc:407] Connection header received
0807 22:46:47.997571 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:47.997575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:47.997618 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:47.997664 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:47.998573 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:47.999106 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:47.999950 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:48.000222 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:48.000310 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:48.002126 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:48.002129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:48.002132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:48.002157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:48.007535 (+ 5378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:48.008101 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:48.008104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:48.008106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:48.008189 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:48.008497 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:48.008500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:48.008502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:48.008696 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:48.008822 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:48.009107 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 22:46:48.009210 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":40}
I20260807 22:46:48.010334 26265 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:48.011200 26265 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:48.015082 26265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f41febb6fb54a4993995e3e269d19d9: Generated new TSK 0
I20260807 22:46:48.015769 26265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:48.023082 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 26142
W20260807 22:46:48.037089 26122 connection.cc:535] client connection to 127.0.0.1:40381 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40381): Connection reset by peer (error 104)
2026-08-07T22:46:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:48 dist-test-slave-4tdj krb5kdc[26291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:48 dist-test-slave-4tdj krb5kdc[26291](info): set up 2 sockets
Aug 07 22:46:48 dist-test-slave-4tdj krb5kdc[26291](info): commencing operation
krb5kdc: starting...
W20260807 22:46:49.173096 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:49 dist-test-slave-4tdj krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:49.291034 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:49.397099 26307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:49.397337 26307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:49.397377 26307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:49.401300 26307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:49.401360 26307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:49.401377 26307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:49.401391 26307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:49.401405 26307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:49.406498 26307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:49.407434 26307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:49.408380 26307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:49.416941 26314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:49.416949 26313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:49.416956 26317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:49.417006 26307 server_base.cc:1034] running on GCE node
I20260807 22:46:49.417671 26307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:49.418771 26307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:49.419919 26307 hybrid_clock.cc:648] HybridClock initialized: now 1786142809419900 us; error 37 us; skew 500 ppm
Aug 07 22:46:49 dist-test-slave-4tdj krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:49.422546 26307 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:49.423421 26307 webserver.cc:460] Webserver started at http://127.0.0.1:34197/ using document root <none> and password file <none>
I20260807 22:46:49.423941 26307 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:49.424007 26307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:49.424196 26307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:49.426177 26307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "5d5c79a8b53a42338bfe22b621618517"
format_stamp: "Formatted at 2026-08-07 22:46:49 on dist-test-slave-4tdj"
server_key: "105f1e29f2e53b4ba51b9680eb56c1d2"
server_key_iv: "9f7852317d986a9f0cbc899ebde0c6ab"
server_key_version: "encryptionkey@0"
I20260807 22:46:49.426755 26307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "5d5c79a8b53a42338bfe22b621618517"
format_stamp: "Formatted at 2026-08-07 22:46:49 on dist-test-slave-4tdj"
server_key: "105f1e29f2e53b4ba51b9680eb56c1d2"
server_key_iv: "9f7852317d986a9f0cbc899ebde0c6ab"
server_key_version: "encryptionkey@0"
I20260807 22:46:49.430269 26307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:46:49.432590 26327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:49.433753 26307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:49.433867 26307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "5d5c79a8b53a42338bfe22b621618517"
format_stamp: "Formatted at 2026-08-07 22:46:49 on dist-test-slave-4tdj"
server_key: "105f1e29f2e53b4ba51b9680eb56c1d2"
server_key_iv: "9f7852317d986a9f0cbc899ebde0c6ab"
server_key_version: "encryptionkey@0"
I20260807 22:46:49.433956 26307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:49.460955 26307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:49.464148 26307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:49.464334 26307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:49.471446 26307 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38153
I20260807 22:46:49.471513 26405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38153 every 8 connection(s)
I20260807 22:46:49.472391 26307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:49.475169 26407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:49.476706 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 26307
I20260807 22:46:49.476845 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:49.477111 27383 external_mini_cluster.cc:1428] Setting key 3a753403d8cf11618f31bcaac17cebf8
I20260807 22:46:49.481420 26407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517: Bootstrap starting.
I20260807 22:46:49.483671 26407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:49.484409 26407 log.cc:824] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:49.485584 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:49.478395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47335 (local address 127.0.0.1:38153)
0807 22:46:49.478821 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:49.478837 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:49.478874 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:46:49.479495 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:49.479512 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:49.479834 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:49.480098 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:49.480531 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:49.481491 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:49.482410 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:49.482877 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:49.483021 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:49.483845 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:49.483851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:49.483864 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:49.484244 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:49.484298 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:49.484321 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:49.484439 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:49.484619 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:46:49.484749 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":106,"threads_started":1}
I20260807 22:46:49.486425 26407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517: No bootstrap required, opened a new log
I20260807 22:46:49.488540 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:49.490850 26407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c79a8b53a42338bfe22b621618517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38153 } }
I20260807 22:46:49.491144 26407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:49.491217 26407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5c79a8b53a42338bfe22b621618517, State: Initialized, Role: FOLLOWER
I20260807 22:46:49.491688 26407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c79a8b53a42338bfe22b621618517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38153 } }
I20260807 22:46:49.491869 26407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:49.491946 26407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:49.492062 26407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:49.493146 26407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c79a8b53a42338bfe22b621618517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38153 } }
I20260807 22:46:49.493521 26407 leader_election.cc:302] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d5c79a8b53a42338bfe22b621618517; no voters:
I20260807 22:46:49.493854 26407 leader_election.cc:288] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:49.493947 26426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:49.494169 26426 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [term 1 LEADER]: Becoming Leader. State: Replica: 5d5c79a8b53a42338bfe22b621618517, State: Running, Role: LEADER
I20260807 22:46:49.494519 26426 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c79a8b53a42338bfe22b621618517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38153 } }
I20260807 22:46:49.495231 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:49.490923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33063 (local address 127.0.0.1:38153)
0807 22:46:49.491027 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:49.491030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:49.491187 (+ 157us) server_negotiation.cc:407] Connection header received
0807 22:46:49.491365 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:49.491368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:49.491408 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:49.491484 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:49.491784 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:49.492402 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:49.493483 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:49.493760 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:49.493877 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:49.494806 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:49.494810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:49.494813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:49.494825 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:49.494852 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:49.494864 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:49.494942 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:49.495059 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 22:46:49.495091 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:46:49.495262 26407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:49.495728 26428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d5c79a8b53a42338bfe22b621618517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c79a8b53a42338bfe22b621618517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38153 } } }
I20260807 22:46:49.495855 26428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:49.496572 26427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d5c79a8b53a42338bfe22b621618517. Latest consensus state: current_term: 1 leader_uuid: "5d5c79a8b53a42338bfe22b621618517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5c79a8b53a42338bfe22b621618517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38153 } } }
I20260807 22:46:49.496667 26427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517 [sys.catalog]: This master's current role is: LEADER
W20260807 22:46:49.500070 26449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:46:49.500165 26449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:46:49.500236 26432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:49.500932 26432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:49.505744 26432 catalog_manager.cc:1347] Generated new cluster ID: 4c5bd8bec77e45bdb3f1bc8614fd6e75
I20260807 22:46:49.505807 26432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:49.516387 26432 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:49.517243 26432 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:49.523118 26432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d5c79a8b53a42338bfe22b621618517: Generated new TSK 0
I20260807 22:46:49.523763 26432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:49.529608 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 26307
W20260807 22:46:49.537832 26283 connection.cc:535] client connection to 127.0.0.1:38153 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38153): Connection reset by peer (error 104)
2026-08-07T22:46:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:49 dist-test-slave-4tdj krb5kdc[26459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:49 dist-test-slave-4tdj krb5kdc[26459](info): set up 2 sockets
Aug 07 22:46:49 dist-test-slave-4tdj krb5kdc[26459](info): commencing operation
krb5kdc: starting...
W20260807 22:46:50.653431 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:50 dist-test-slave-4tdj krb5kdc[26459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:50.769290 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:50.877859 26486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:50.878103 26486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:50.878144 26486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:50.882020 26486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:50.882081 26486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:50.882102 26486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:50.882120 26486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:50.882138 26486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:50.887602 26486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:50.888638 26486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:50.889626 26486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:50.897926 26492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:50.897914 26493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:50.898025 26486 server_base.cc:1034] running on GCE node
W20260807 22:46:50.897927 26495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:50.898702 26486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:50.899775 26486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:50.900923 26486 hybrid_clock.cc:648] HybridClock initialized: now 1786142810900899 us; error 37 us; skew 500 ppm
Aug 07 22:46:50 dist-test-slave-4tdj krb5kdc[26459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:50.903566 26486 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:50.904398 26486 webserver.cc:460] Webserver started at http://127.0.0.1:39683/ using document root <none> and password file <none>
I20260807 22:46:50.904906 26486 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:50.904964 26486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:50.905169 26486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:50.907171 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "0238f26c5fb8430b87cd3ba2da7c0eda"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-4tdj"
server_key: "1d869e41ec2e4c7e517d60e0ac36e0a9"
server_key_iv: "6e5dadd88763bc762ded499c8cb96af8"
server_key_version: "encryptionkey@0"
I20260807 22:46:50.907701 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "0238f26c5fb8430b87cd3ba2da7c0eda"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-4tdj"
server_key: "1d869e41ec2e4c7e517d60e0ac36e0a9"
server_key_iv: "6e5dadd88763bc762ded499c8cb96af8"
server_key_version: "encryptionkey@0"
I20260807 22:46:50.911377 26486 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:46:50.913817 26513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:50.915042 26486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:50.915167 26486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "0238f26c5fb8430b87cd3ba2da7c0eda"
format_stamp: "Formatted at 2026-08-07 22:46:50 on dist-test-slave-4tdj"
server_key: "1d869e41ec2e4c7e517d60e0ac36e0a9"
server_key_iv: "6e5dadd88763bc762ded499c8cb96af8"
server_key_version: "encryptionkey@0"
I20260807 22:46:50.915266 26486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:50.919835 26486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:50.922668 26486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:50.922827 26486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:50.929325 26486 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39331
I20260807 22:46:50.929389 26603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39331 every 8 connection(s)
I20260807 22:46:50.930181 26486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:50.932811 26604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:50.934648 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 26486
I20260807 22:46:50.934793 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:50.935110 27383 external_mini_cluster.cc:1428] Setting key 37acb46bc60466547b574aca861cca83
I20260807 22:46:50.939285 26604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda: Bootstrap starting.
I20260807 22:46:50.939996 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:50.936405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41505 (local address 127.0.0.1:39331)
0807 22:46:50.936804 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:50.936819 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:50.936853 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:46:50.937467 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:50.937483 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:50.937738 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:50.938000 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:50.938799 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:50.938804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:50.938815 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:50.938824 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:50.938875 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:50.938893 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:50.939004 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:50.939099 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 22:46:50.939234 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":122,"threads_started":1}
I20260807 22:46:50.941642 26604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:50.942449 26604 log.cc:824] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:50.942662 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:50.944213 26604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda: No bootstrap required, opened a new log
I20260807 22:46:50.945928 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:50.944567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45679 (local address 127.0.0.1:39331)
0807 22:46:50.944677 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:50.944681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:50.944814 (+ 133us) server_negotiation.cc:407] Connection header received
0807 22:46:50.944929 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:50.944931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:50.944956 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:50.945016 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:50.945622 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:50.945625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:50.945627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:50.945636 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:50.945659 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:50.945669 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:50.945730 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:50.945806 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 22:46:50.945832 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:46:50.948083 26604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39331 } }
I20260807 22:46:50.948290 26604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:50.948355 26604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0238f26c5fb8430b87cd3ba2da7c0eda, State: Initialized, Role: FOLLOWER
I20260807 22:46:50.948740 26604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39331 } }
I20260807 22:46:50.948864 26604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:50.948925 26604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:50.949025 26604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:50.949842 26604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39331 } }
I20260807 22:46:50.950161 26604 leader_election.cc:302] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0238f26c5fb8430b87cd3ba2da7c0eda; no voters:
I20260807 22:46:50.950474 26604 leader_election.cc:288] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:50.950594 26622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:50.950825 26622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [term 1 LEADER]: Becoming Leader. State: Replica: 0238f26c5fb8430b87cd3ba2da7c0eda, State: Running, Role: LEADER
I20260807 22:46:50.951115 26622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39331 } }
I20260807 22:46:50.951634 26604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:50.952167 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0238f26c5fb8430b87cd3ba2da7c0eda. Latest consensus state: current_term: 1 leader_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39331 } } }
I20260807 22:46:50.952346 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:50.952574 26624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0238f26c5fb8430b87cd3ba2da7c0eda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39331 } } }
I20260807 22:46:50.952678 26624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:50.952877 26631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:50.955708 26631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:50.960975 26631 catalog_manager.cc:1347] Generated new cluster ID: 987343e711c34ba6b4b083e64e165f22
I20260807 22:46:50.961048 26631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:50.968873 26631 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:50.969136 26631 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:50.972944 26631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0238f26c5fb8430b87cd3ba2da7c0eda: Generated new TSK 0
I20260807 22:46:50.973488 26631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:50.979182 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 26486
2026-08-07T22:46:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:51 dist-test-slave-4tdj krb5kdc[26656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:51 dist-test-slave-4tdj krb5kdc[26656](info): set up 2 sockets
Aug 07 22:46:51 dist-test-slave-4tdj krb5kdc[26656](info): commencing operation
krb5kdc: starting...
W20260807 22:46:52.123919 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:52 dist-test-slave-4tdj krb5kdc[26656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:52.239058 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:52.347195 26676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:52.347461 26676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:52.347522 26676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:52.351508 26676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:52.351599 26676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:52.351640 26676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:52.351671 26676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:52.351699 26676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:52.356881 26676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:52.357909 26676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:52.358917 26676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:52.366988 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:52.366989 26683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:52.367026 26684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:52.367276 26676 server_base.cc:1034] running on GCE node
I20260807 22:46:52.367836 26676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:52.368947 26676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:52.370107 26676 hybrid_clock.cc:648] HybridClock initialized: now 1786142812370094 us; error 29 us; skew 500 ppm
Aug 07 22:46:52 dist-test-slave-4tdj krb5kdc[26656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142812, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:52.372835 26676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:52.373718 26676 webserver.cc:460] Webserver started at http://127.0.0.1:34067/ using document root <none> and password file <none>
I20260807 22:46:52.374249 26676 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:52.374343 26676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:52.374552 26676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:52.376570 26676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "0555f129b48e4dc5810b3c485042c739"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-4tdj"
server_key: "714a0102fae6c922f95cbf8522790ed4"
server_key_iv: "de0e04ac1dd058f7646e808ce3fee5de"
server_key_version: "encryptionkey@0"
I20260807 22:46:52.377136 26676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "0555f129b48e4dc5810b3c485042c739"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-4tdj"
server_key: "714a0102fae6c922f95cbf8522790ed4"
server_key_iv: "de0e04ac1dd058f7646e808ce3fee5de"
server_key_version: "encryptionkey@0"
I20260807 22:46:52.380734 26676 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:46:52.383068 26700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:52.384286 26676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:46:52.384409 26676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "0555f129b48e4dc5810b3c485042c739"
format_stamp: "Formatted at 2026-08-07 22:46:52 on dist-test-slave-4tdj"
server_key: "714a0102fae6c922f95cbf8522790ed4"
server_key_iv: "de0e04ac1dd058f7646e808ce3fee5de"
server_key_version: "encryptionkey@0"
I20260807 22:46:52.384503 26676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:52.397876 26676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:52.400902 26676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:52.401098 26676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:52.408201 26676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42477
I20260807 22:46:52.408219 26775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42477 every 8 connection(s)
I20260807 22:46:52.409121 26676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:52.411813 26776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:52.415040 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 26676
I20260807 22:46:52.415198 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:52.415495 27383 external_mini_cluster.cc:1428] Setting key 5b602b28d0cce308d37695af085324fe
I20260807 22:46:52.417694 26776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739: Bootstrap starting.
I20260807 22:46:52.420113 26776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:52.420948 26776 log.cc:824] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:52.423087 26776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739: No bootstrap required, opened a new log
Aug 07 22:46:52 dist-test-slave-4tdj krb5kdc[26656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:46:52.426920 26776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555f129b48e4dc5810b3c485042c739" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 22:46:52.427191 26776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:52.427246 26776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0555f129b48e4dc5810b3c485042c739, State: Initialized, Role: FOLLOWER
I20260807 22:46:52.427683 26776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555f129b48e4dc5810b3c485042c739" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 22:46:52.427814 26776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:52.427875 26776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:52.427969 26776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:52.428871 26776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555f129b48e4dc5810b3c485042c739" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 22:46:52.429178 26776 leader_election.cc:302] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0555f129b48e4dc5810b3c485042c739; no voters:
I20260807 22:46:52.429447 26776 leader_election.cc:288] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:52.429533 26787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:52.429661 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.416813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54017 (local address 127.0.0.1:42477)
0807 22:46:52.417195 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.417211 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.417246 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:46:52.417878 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.417896 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.418228 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.418523 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:52.419378 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.420379 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.421490 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.422028 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.422149 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.424976 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:52.424981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:52.424994 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:52.425027 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:52.426671 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.427135 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.427140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.427144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.427286 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.427559 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.427562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.427564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.428006 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:52.428210 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.428496 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.428770 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
I20260807 22:46:52.429702 26787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [term 1 LEADER]: Becoming Leader. State: Replica: 0555f129b48e4dc5810b3c485042c739, State: Running, Role: LEADER
I20260807 22:46:52.430270 26787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555f129b48e4dc5810b3c485042c739" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 22:46:52.430720 26776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:52.432951 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:52.432744 26790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0555f129b48e4dc5810b3c485042c739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555f129b48e4dc5810b3c485042c739" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } } }
I20260807 22:46:52.433157 26790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:52.432935 26792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0555f129b48e4dc5810b3c485042c739. Latest consensus state: current_term: 1 leader_uuid: "0555f129b48e4dc5810b3c485042c739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555f129b48e4dc5810b3c485042c739" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } } }
I20260807 22:46:52.433346 26792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739 [sys.catalog]: This master's current role is: LEADER
W20260807 22:46:52.435797 26809 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:46:52.435876 26809 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:46:52.435951 26810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:52.436841 26810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:52.442554 26810 catalog_manager.cc:1347] Generated new cluster ID: 4445539cacd141da909617a3e785026c
I20260807 22:46:52.442616 26810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:52.445868 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:52.435236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34221 (local address 127.0.0.1:42477)
0807 22:46:52.435335 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:52.435339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:52.435632 (+ 293us) server_negotiation.cc:407] Connection header received
0807 22:46:52.435733 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:52.435736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:52.435781 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:52.435864 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:52.439162 (+ 3298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.439985 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.440998 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:52.441365 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:52.441475 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:52.443089 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:52.443093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:52.443096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:52.443121 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:52.444375 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.444758 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.444761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.444763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.444835 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:52.445077 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:52.445080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:52.445082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:52.445268 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:52.445380 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:52.445619 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 22:46:52.445726 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:46:52.465531 26810 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:52.467041 26810 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:52.476505 26810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0555f129b48e4dc5810b3c485042c739: Generated new TSK 0
I20260807 22:46:52.477175 26810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:52.487938 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 26676
2026-08-07T22:46:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:52 dist-test-slave-4tdj krb5kdc[26825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:52 dist-test-slave-4tdj krb5kdc[26825](info): set up 2 sockets
Aug 07 22:46:52 dist-test-slave-4tdj krb5kdc[26825](info): commencing operation
krb5kdc: starting...
W20260807 22:46:53.634093 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:53 dist-test-slave-4tdj krb5kdc[26825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:53.753311 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:53.863127 26850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:53.863368 26850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:53.863406 26850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:53.867272 26850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:53.867322 26850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:53.867341 26850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:53.867355 26850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:53.867369 26850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:53.872464 26850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:53.873399 26850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:53.874344 26850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:53.882198 26857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:53.882190 26858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:53.882277 26860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:53.882279 26850 server_base.cc:1034] running on GCE node
I20260807 22:46:53.882975 26850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:53.884003 26850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:53.885159 26850 hybrid_clock.cc:648] HybridClock initialized: now 1786142813885135 us; error 41 us; skew 500 ppm
Aug 07 22:46:53 dist-test-slave-4tdj krb5kdc[26825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:53.892256 26850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:53.893338 26850 webserver.cc:460] Webserver started at http://127.0.0.1:46121/ using document root <none> and password file <none>
I20260807 22:46:53.893877 26850 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:53.893944 26850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:53.894145 26850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:53.896181 26850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "4ec9913c72f242f490118d7981d61827"
format_stamp: "Formatted at 2026-08-07 22:46:53 on dist-test-slave-4tdj"
server_key: "b67f4dc947d1337dac940f7149473059"
server_key_iv: "1b53dfe7bbad3974415b95dd335ae45d"
server_key_version: "encryptionkey@0"
I20260807 22:46:53.896727 26850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "4ec9913c72f242f490118d7981d61827"
format_stamp: "Formatted at 2026-08-07 22:46:53 on dist-test-slave-4tdj"
server_key: "b67f4dc947d1337dac940f7149473059"
server_key_iv: "1b53dfe7bbad3974415b95dd335ae45d"
server_key_version: "encryptionkey@0"
I20260807 22:46:53.900357 26850 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 22:46:53.902694 26872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:53.903993 26850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:46:53.904137 26850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "4ec9913c72f242f490118d7981d61827"
format_stamp: "Formatted at 2026-08-07 22:46:53 on dist-test-slave-4tdj"
server_key: "b67f4dc947d1337dac940f7149473059"
server_key_iv: "1b53dfe7bbad3974415b95dd335ae45d"
server_key_version: "encryptionkey@0"
I20260807 22:46:53.904232 26850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:53.914661 26850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:53.917294 26850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:53.917459 26850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:53.924481 26850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35859
I20260807 22:46:53.924540 26958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35859 every 8 connection(s)
I20260807 22:46:53.925396 26850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:53.928076 26959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:53.929517 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 26850
I20260807 22:46:53.929668 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:53.929963 27383 external_mini_cluster.cc:1428] Setting key 9c5567e36dfb195786be255b636d1a73
I20260807 22:46:53.934088 26959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827: Bootstrap starting.
I20260807 22:46:53.936662 26959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:53.937423 26959 log.cc:824] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:53.938642 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:53.931264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49623 (local address 127.0.0.1:35859)
0807 22:46:53.931652 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:53.931668 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:53.931703 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:46:53.932323 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:53.932339 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:53.932667 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:53.932927 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:53.933305 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.934360 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.935536 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.936004 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.936125 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:53.936938 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:53.936944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:53.936956 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:53.937322 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:53.937375 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:53.937397 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:53.937468 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:53.937658 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 22:46:53.937866 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":106,"threads_started":1}
I20260807 22:46:53.939368 26959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827: No bootstrap required, opened a new log
I20260807 22:46:53.941452 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:53.943298 26959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9913c72f242f490118d7981d61827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35859 } }
I20260807 22:46:53.943563 26959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:53.943636 26959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec9913c72f242f490118d7981d61827, State: Initialized, Role: FOLLOWER
I20260807 22:46:53.944144 26959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9913c72f242f490118d7981d61827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35859 } }
I20260807 22:46:53.944310 26959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:53.944379 26959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:53.944478 26959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:53.945546 26959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9913c72f242f490118d7981d61827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35859 } }
I20260807 22:46:53.945866 26959 leader_election.cc:302] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ec9913c72f242f490118d7981d61827; no voters:
I20260807 22:46:53.946161 26959 leader_election.cc:288] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:53.946236 26976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:53.946455 26976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [term 1 LEADER]: Becoming Leader. State: Replica: 4ec9913c72f242f490118d7981d61827, State: Running, Role: LEADER
I20260807 22:46:53.946762 26976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9913c72f242f490118d7981d61827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35859 } }
I20260807 22:46:53.947324 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:53.943423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36765 (local address 127.0.0.1:35859)
0807 22:46:53.943580 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:53.943585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:53.943723 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:46:53.943816 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:53.943820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:53.943867 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:53.943948 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:53.944212 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.944934 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.945739 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:53.946021 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:53.946116 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:53.946925 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:53.946929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:53.946931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:53.946942 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:53.946970 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:53.946983 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:53.947053 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:53.947152 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 22:46:53.947202 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:46:53.947494 26959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:53.947973 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ec9913c72f242f490118d7981d61827. Latest consensus state: current_term: 1 leader_uuid: "4ec9913c72f242f490118d7981d61827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9913c72f242f490118d7981d61827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35859 } } }
I20260807 22:46:53.948100 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:53.948347 26979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ec9913c72f242f490118d7981d61827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec9913c72f242f490118d7981d61827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35859 } } }
I20260807 22:46:53.948449 26979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827 [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:53.948884 26983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:53.952131 26983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:53.957727 26983 catalog_manager.cc:1347] Generated new cluster ID: ceffbfbde9f84f1aa2b0bdda857997d8
I20260807 22:46:53.957796 26983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:53.996513 26983 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:53.997810 26983 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:54.008946 26983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ec9913c72f242f490118d7981d61827: Generated new TSK 0
I20260807 22:46:54.009734 26983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:54.022552 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 26850
2026-08-07T22:46:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:54 dist-test-slave-4tdj krb5kdc[27003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:54 dist-test-slave-4tdj krb5kdc[27003](info): set up 2 sockets
Aug 07 22:46:54 dist-test-slave-4tdj krb5kdc[27003](info): commencing operation
krb5kdc: starting...
W20260807 22:46:55.150753 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:55 dist-test-slave-4tdj krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:55.266357 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:55.372531 27021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:55.372750 27021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:55.372792 27021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:55.376631 27021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:55.376677 27021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:55.376695 27021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:55.376709 27021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:55.376729 27021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:55.381692 27021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:55.382594 27021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:55.383507 27021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:55.391392 27029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:55.391418 27028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:55.391403 27033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:55.391741 27021 server_base.cc:1034] running on GCE node
I20260807 22:46:55.392233 27021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:55.393230 27021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:55.394352 27021 hybrid_clock.cc:648] HybridClock initialized: now 1786142815394332 us; error 33 us; skew 500 ppm
Aug 07 22:46:55 dist-test-slave-4tdj krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:55.396916 27021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:55.397773 27021 webserver.cc:460] Webserver started at http://127.0.0.1:38133/ using document root <none> and password file <none>
I20260807 22:46:55.398263 27021 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:55.398360 27021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:55.398540 27021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:55.400403 27021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "c0aa2444bf274e5780615867913644fc"
format_stamp: "Formatted at 2026-08-07 22:46:55 on dist-test-slave-4tdj"
server_key: "5a73507a12a213f4d27803fbfd26065c"
server_key_iv: "9d79ada3524b1f76a9a947db5d3fc87b"
server_key_version: "encryptionkey@0"
I20260807 22:46:55.400890 27021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "c0aa2444bf274e5780615867913644fc"
format_stamp: "Formatted at 2026-08-07 22:46:55 on dist-test-slave-4tdj"
server_key: "5a73507a12a213f4d27803fbfd26065c"
server_key_iv: "9d79ada3524b1f76a9a947db5d3fc87b"
server_key_version: "encryptionkey@0"
I20260807 22:46:55.404369 27021 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:46:55.406603 27045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:55.407847 27021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:46:55.407979 27021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "c0aa2444bf274e5780615867913644fc"
format_stamp: "Formatted at 2026-08-07 22:46:55 on dist-test-slave-4tdj"
server_key: "5a73507a12a213f4d27803fbfd26065c"
server_key_iv: "9d79ada3524b1f76a9a947db5d3fc87b"
server_key_version: "encryptionkey@0"
I20260807 22:46:55.408087 27021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:55.412439 27021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:55.415371 27021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:55.415525 27021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:55.423734 27021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45121
I20260807 22:46:55.423779 27120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45121 every 8 connection(s)
I20260807 22:46:55.424595 27021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:55.427291 27121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:55.431694 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27021
I20260807 22:46:55.431834 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:55.432108 27383 external_mini_cluster.cc:1428] Setting key 70597a50388839def85229d1d70c2c76
I20260807 22:46:55.432478 27121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc: Bootstrap starting.
I20260807 22:46:55.434988 27121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:55.435878 27121 log.cc:824] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:55.437415 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.433391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39725 (local address 127.0.0.1:45121)
0807 22:46:55.433823 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.433841 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.433883 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:46:55.434697 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.434719 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.435071 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.435427 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:55.436049 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:55.436055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:55.436069 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:55.436080 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:55.436139 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:55.436166 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:55.436271 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.436383 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.436518 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":114,"threads_started":1}
I20260807 22:46:55.438092 27121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc: No bootstrap required, opened a new log
I20260807 22:46:55.440016 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:55.441772 27121 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa2444bf274e5780615867913644fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260807 22:46:55.442025 27121 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:55.442094 27121 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0aa2444bf274e5780615867913644fc, State: Initialized, Role: FOLLOWER
I20260807 22:46:55.442564 27121 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa2444bf274e5780615867913644fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260807 22:46:55.442713 27121 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:55.442780 27121 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:55.442878 27121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:55.443418 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:55.441935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54749 (local address 127.0.0.1:45121)
0807 22:46:55.442128 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:55.442132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:55.442203 (+ 71us) server_negotiation.cc:407] Connection header received
0807 22:46:55.442284 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:55.442287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:55.442341 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:55.442413 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:55.443092 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:55.443096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:55.443098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:55.443110 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:55.443137 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:55.443149 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:55.443231 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:55.443279 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 22:46:55.443312 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:46:55.443885 27121 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa2444bf274e5780615867913644fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260807 22:46:55.444188 27121 leader_election.cc:302] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0aa2444bf274e5780615867913644fc; no voters:
I20260807 22:46:55.444446 27121 leader_election.cc:288] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:55.444571 27135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:55.444799 27135 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [term 1 LEADER]: Becoming Leader. State: Replica: c0aa2444bf274e5780615867913644fc, State: Running, Role: LEADER
I20260807 22:46:55.445065 27135 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa2444bf274e5780615867913644fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260807 22:46:55.445556 27121 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:55.446595 27137 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0aa2444bf274e5780615867913644fc. Latest consensus state: current_term: 1 leader_uuid: "c0aa2444bf274e5780615867913644fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa2444bf274e5780615867913644fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } } }
I20260807 22:46:55.446724 27137 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:55.446986 27136 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0aa2444bf274e5780615867913644fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa2444bf274e5780615867913644fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } } }
I20260807 22:46:55.447094 27136 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:55.447319 27146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:55.449903 27146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:55.454560 27146 catalog_manager.cc:1347] Generated new cluster ID: 9e2456f20e4f42738e3e2fbb25838d70
I20260807 22:46:55.454618 27146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:55.479254 27146 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:55.479581 27146 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:55.487241 27146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0aa2444bf274e5780615867913644fc: Generated new TSK 0
I20260807 22:46:55.487887 27146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:55.493476 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27021
2026-08-07T22:46:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:55 dist-test-slave-4tdj krb5kdc[27161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:55 dist-test-slave-4tdj krb5kdc[27161](info): set up 2 sockets
Aug 07 22:46:55 dist-test-slave-4tdj krb5kdc[27161](info): commencing operation
krb5kdc: starting...
W20260807 22:46:56.630849 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:56 dist-test-slave-4tdj krb5kdc[27161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:56.749657 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:56.856722 27179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:56.856962 27179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:56.857018 27179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:56.860961 27179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:56.861016 27179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:56.861042 27179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:56.861063 27179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:56.861088 27179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:56.866113 27179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:56.867110 27179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:56.868063 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:56.876102 27196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:56.876160 27192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:56.876279 27179 server_base.cc:1034] running on GCE node
W20260807 22:46:56.876161 27198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:56.876892 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:56.877918 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:56.879091 27179 hybrid_clock.cc:648] HybridClock initialized: now 1786142816879069 us; error 34 us; skew 500 ppm
Aug 07 22:46:56 dist-test-slave-4tdj krb5kdc[27161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:56.881770 27179 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:56.882647 27179 webserver.cc:460] Webserver started at http://127.0.0.1:41687/ using document root <none> and password file <none>
I20260807 22:46:56.883164 27179 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:56.883229 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:56.883412 27179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:56.885460 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "9de66888e357480cac67331d181b921b"
format_stamp: "Formatted at 2026-08-07 22:46:56 on dist-test-slave-4tdj"
server_key: "45aa2d04bb27dffbf81ce8c3aff3a041"
server_key_iv: "0dc4fba955c0bfa774382fd16b07f561"
server_key_version: "encryptionkey@0"
I20260807 22:46:56.885998 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "9de66888e357480cac67331d181b921b"
format_stamp: "Formatted at 2026-08-07 22:46:56 on dist-test-slave-4tdj"
server_key: "45aa2d04bb27dffbf81ce8c3aff3a041"
server_key_iv: "0dc4fba955c0bfa774382fd16b07f561"
server_key_version: "encryptionkey@0"
I20260807 22:46:56.889587 27179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:46:56.891904 27207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:56.893069 27179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:46:56.893181 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "9de66888e357480cac67331d181b921b"
format_stamp: "Formatted at 2026-08-07 22:46:56 on dist-test-slave-4tdj"
server_key: "45aa2d04bb27dffbf81ce8c3aff3a041"
server_key_iv: "0dc4fba955c0bfa774382fd16b07f561"
server_key_version: "encryptionkey@0"
I20260807 22:46:56.893268 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:56.916252 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:56.919323 27179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:56.919499 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:56.926724 27179 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33259
I20260807 22:46:56.926774 27283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33259 every 8 connection(s)
I20260807 22:46:56.927675 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:56.930413 27285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:56.935614 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27179
I20260807 22:46:56.935742 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:56.935796 27285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b: Bootstrap starting.
I20260807 22:46:56.936018 27383 external_mini_cluster.cc:1428] Setting key 6f80072e910df5d1d236c2e985d98a6b
I20260807 22:46:56.938241 27285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:56.939013 27285 log.cc:824] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:56.940939 27285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b: No bootstrap required, opened a new log
Aug 07 22:46:56 dist-test-slave-4tdj krb5kdc[27161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:46:56.945103 27285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de66888e357480cac67331d181b921b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260807 22:46:56.945335 27285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:56.945393 27285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de66888e357480cac67331d181b921b, State: Initialized, Role: FOLLOWER
I20260807 22:46:56.945779 27285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de66888e357480cac67331d181b921b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260807 22:46:56.945922 27285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:56.946002 27285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:56.946105 27285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:56.947042 27285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de66888e357480cac67331d181b921b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260807 22:46:56.947351 27285 leader_election.cc:302] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9de66888e357480cac67331d181b921b; no voters:
I20260807 22:46:56.947671 27285 leader_election.cc:288] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:56.947749 27291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:56.947953 27291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [term 1 LEADER]: Becoming Leader. State: Replica: 9de66888e357480cac67331d181b921b, State: Running, Role: LEADER
I20260807 22:46:56.948253 27291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de66888e357480cac67331d181b921b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260807 22:46:56.948962 27285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:56.950142 27293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9de66888e357480cac67331d181b921b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de66888e357480cac67331d181b921b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } } }
I20260807 22:46:56.950285 27293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:56.950632 27294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9de66888e357480cac67331d181b921b. Latest consensus state: current_term: 1 leader_uuid: "9de66888e357480cac67331d181b921b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de66888e357480cac67331d181b921b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } } }
I20260807 22:46:56.950726 27294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:56.950841 27306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:56.950901 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.937441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:56283 (local address 127.0.0.1:33259)
0807 22:46:56.937826 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.937854 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.937888 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:46:56.938587 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.938607 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.938951 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.939239 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:56.940033 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.940986 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.942165 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.942671 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.942787 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.945530 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:56.945535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:56.945546 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:56.945573 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:56.947331 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.947951 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.947958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.947964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.948100 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.948518 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.948521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.948523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.949000 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:56.949255 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.949593 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.949835 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":107,"threads_started":1}
I20260807 22:46:56.954041 27306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:56.954408 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:56.961009 27306 catalog_manager.cc:1347] Generated new cluster ID: 878a63bb390540edb3fcafbb978f63ff
I20260807 22:46:56.961074 27306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:56.964357 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:56.956696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:48555 (local address 127.0.0.1:33259)
0807 22:46:56.956794 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:56.956797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:56.957088 (+ 291us) server_negotiation.cc:407] Connection header received
0807 22:46:56.957193 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:56.957196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:56.957238 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:56.957300 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:56.958085 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.958639 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.959434 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:56.959669 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:56.959754 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:56.961043 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:56.961047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:56.961049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:46:56.961075 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:56.962665 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.963184 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.963187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.963189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.963264 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:46:56.963575 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:46:56.963577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:46:56.963579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:46:56.963754 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:56.963873 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:56.964127 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 22:46:56.964216 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:46:56.969019 27306 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:56.969825 27306 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:56.986768 27306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9de66888e357480cac67331d181b921b: Generated new TSK 0
I20260807 22:46:56.987558 27306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:56.998735 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27179
W20260807 22:46:57.015260 27159 connection.cc:535] client connection to 127.0.0.1:33259 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33259): Connection reset by peer (error 104)
2026-08-07T22:46:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:57 dist-test-slave-4tdj krb5kdc[27328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:57 dist-test-slave-4tdj krb5kdc[27328](info): set up 2 sockets
Aug 07 22:46:57 dist-test-slave-4tdj krb5kdc[27328](info): commencing operation
krb5kdc: starting...
W20260807 22:46:58.145872 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:58 dist-test-slave-4tdj krb5kdc[27328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:58.264892 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:58.372110 27345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:58.372349 27345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:58.372387 27345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:58.376240 27345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:58.376291 27345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:58.376308 27345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:58.376322 27345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:58.376336 27345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:58.381287 27345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:58.382207 27345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:58.383184 27345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:58.391098 27359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:58.391179 27362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:58.391186 27357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:58.391419 27345 server_base.cc:1034] running on GCE node
I20260807 22:46:58.391928 27345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:58.392951 27345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:58.394124 27345 hybrid_clock.cc:648] HybridClock initialized: now 1786142818394098 us; error 35 us; skew 500 ppm
Aug 07 22:46:58 dist-test-slave-4tdj krb5kdc[27328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:58.396852 27345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:58.397711 27345 webserver.cc:460] Webserver started at http://127.0.0.1:44147/ using document root <none> and password file <none>
I20260807 22:46:58.398214 27345 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:58.398281 27345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:58.398540 27345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:58.400501 27345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "bf5ce75a6e35445183ef81a873c6e7cc"
format_stamp: "Formatted at 2026-08-07 22:46:58 on dist-test-slave-4tdj"
server_key: "e71adf57c82bcc70c75eb133136de9a0"
server_key_iv: "9e7b526b50dfdc75ce6fc80f65a84626"
server_key_version: "encryptionkey@0"
I20260807 22:46:58.401044 27345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "bf5ce75a6e35445183ef81a873c6e7cc"
format_stamp: "Formatted at 2026-08-07 22:46:58 on dist-test-slave-4tdj"
server_key: "e71adf57c82bcc70c75eb133136de9a0"
server_key_iv: "9e7b526b50dfdc75ce6fc80f65a84626"
server_key_version: "encryptionkey@0"
I20260807 22:46:58.404574 27345 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 22:46:58.406906 27372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:58.408057 27345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:46:58.408170 27345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "bf5ce75a6e35445183ef81a873c6e7cc"
format_stamp: "Formatted at 2026-08-07 22:46:58 on dist-test-slave-4tdj"
server_key: "e71adf57c82bcc70c75eb133136de9a0"
server_key_iv: "9e7b526b50dfdc75ce6fc80f65a84626"
server_key_version: "encryptionkey@0"
I20260807 22:46:58.408262 27345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:58.420363 27345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:58.423264 27345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:58.423440 27345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:58.430229 27345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39161
I20260807 22:46:58.430241 27458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39161 every 8 connection(s)
I20260807 22:46:58.431159 27345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:58.433791 27459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:58.439200 27459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc: Bootstrap starting.
I20260807 22:46:58.440685 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27345
I20260807 22:46:58.440835 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:58.441162 27383 external_mini_cluster.cc:1428] Setting key cd30f57de201e65aed749b193947c38a
I20260807 22:46:58.441926 27459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:58.442955 27459 log.cc:824] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:58.444916 27459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc: No bootstrap required, opened a new log
I20260807 22:46:58.448959 27459 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39161 } }
I20260807 22:46:58.449221 27459 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:58.449285 27459 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5ce75a6e35445183ef81a873c6e7cc, State: Initialized, Role: FOLLOWER
I20260807 22:46:58.449697 27459 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39161 } }
I20260807 22:46:58.449826 27459 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:58.449891 27459 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:58.449987 27459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:58.450006 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.442360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:44565 (local address 127.0.0.1:39161)
0807 22:46:58.442765 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:58.442781 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:58.442814 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:46:58.443472 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:58.443489 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:58.443819 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:58.444092 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:58.444462 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:58.445462 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:58.446699 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:58.447177 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:58.447293 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:58.448186 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:58.448191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:58.448203 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:58.448596 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:58.448651 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:58.448678 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:58.448784 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:58.448970 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 22:46:58.449099 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":128,"threads_started":1}
I20260807 22:46:58.451061 27459 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39161 } }
I20260807 22:46:58.451387 27459 leader_election.cc:302] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf5ce75a6e35445183ef81a873c6e7cc; no voters:
I20260807 22:46:58.451795 27459 leader_election.cc:288] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:58.451859 27469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:58.452080 27469 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [term 1 LEADER]: Becoming Leader. State: Replica: bf5ce75a6e35445183ef81a873c6e7cc, State: Running, Role: LEADER
I20260807 22:46:58.452423 27469 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39161 } }
I20260807 22:46:58.453162 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:58.453186 27459 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:58.454259 27471 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf5ce75a6e35445183ef81a873c6e7cc. Latest consensus state: current_term: 1 leader_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39161 } } }
I20260807 22:46:58.454403 27471 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:58.454728 27476 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5ce75a6e35445183ef81a873c6e7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39161 } } }
I20260807 22:46:58.454836 27476 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:58.455492 27492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:58.458343 27492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:58.460191 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:58.455497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:39333 (local address 127.0.0.1:39161)
0807 22:46:58.455599 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:58.455604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:58.456067 (+ 463us) server_negotiation.cc:407] Connection header received
0807 22:46:58.456156 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:58.456159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:58.456206 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:58.456292 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:58.456560 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:58.457275 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:58.458422 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:46:58.458746 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:46:58.458851 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:46:58.459734 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:58.459738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:58.459741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:58.459753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:58.459781 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:58.459794 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:58.459879 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:58.460034 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:46:58.460067 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:46:58.463768 27492 catalog_manager.cc:1347] Generated new cluster ID: 3273cab33fd041c593171ed9e2275f1f
I20260807 22:46:58.463830 27492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:58.470959 27492 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:58.471822 27492 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:58.481812 27492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf5ce75a6e35445183ef81a873c6e7cc: Generated new TSK 0
I20260807 22:46:58.482511 27492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:58.486073 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27345
W20260807 22:46:58.503710 27324 connection.cc:535] client connection to 127.0.0.1:39161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39161): Connection reset by peer (error 104)
2026-08-07T22:46:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:46:58 dist-test-slave-4tdj krb5kdc[27513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:46:58 dist-test-slave-4tdj krb5kdc[27513](info): set up 2 sockets
Aug 07 22:46:58 dist-test-slave-4tdj krb5kdc[27513](info): commencing operation
krb5kdc: starting...
W20260807 22:46:59.627417 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:46:59 dist-test-slave-4tdj krb5kdc[27513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:46:59.747040 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:46:59.854946 27533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:46:59.855181 27533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:46:59.855219 27533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:46:59.859117 27533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:46:59.859167 27533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:46:59.859185 27533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:46:59.859200 27533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:46:59.859213 27533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:46:59.864216 27533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:46:59.865407 27533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:46:59.866391 27533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:46:59.874415 27543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:59.874451 27540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:46:59.874416 27538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:46:59.874502 27533 server_base.cc:1034] running on GCE node
I20260807 22:46:59.875212 27533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:46:59.876212 27533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:46:59.877374 27533 hybrid_clock.cc:648] HybridClock initialized: now 1786142819877354 us; error 31 us; skew 500 ppm
Aug 07 22:46:59 dist-test-slave-4tdj krb5kdc[27513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:46:59.883117 27533 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:46:59.883989 27533 webserver.cc:460] Webserver started at http://127.0.0.1:44033/ using document root <none> and password file <none>
I20260807 22:46:59.884508 27533 fs_manager.cc:359] Metadata directory not provided
I20260807 22:46:59.884575 27533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:46:59.884759 27533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:46:59.886817 27533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "a41d4b32cc814f1ba33a9ade4fceefee"
format_stamp: "Formatted at 2026-08-07 22:46:59 on dist-test-slave-4tdj"
server_key: "67826acfa36d367af61639eeb3f3f20b"
server_key_iv: "ee2b70d15d5966b1a9e5c25031d83107"
server_key_version: "encryptionkey@0"
I20260807 22:46:59.887369 27533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "a41d4b32cc814f1ba33a9ade4fceefee"
format_stamp: "Formatted at 2026-08-07 22:46:59 on dist-test-slave-4tdj"
server_key: "67826acfa36d367af61639eeb3f3f20b"
server_key_iv: "ee2b70d15d5966b1a9e5c25031d83107"
server_key_version: "encryptionkey@0"
I20260807 22:46:59.890936 27533 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 22:46:59.893234 27553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:46:59.894495 27533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 22:46:59.894629 27533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "a41d4b32cc814f1ba33a9ade4fceefee"
format_stamp: "Formatted at 2026-08-07 22:46:59 on dist-test-slave-4tdj"
server_key: "67826acfa36d367af61639eeb3f3f20b"
server_key_iv: "ee2b70d15d5966b1a9e5c25031d83107"
server_key_version: "encryptionkey@0"
I20260807 22:46:59.894718 27533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:46:59.899296 27533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:46:59.902177 27533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:46:59.902369 27533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:46:59.909072 27533 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46609
I20260807 22:46:59.909080 27630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46609 every 8 connection(s)
I20260807 22:46:59.909948 27533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:46:59.912659 27631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:46:59.912753 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27533
I20260807 22:46:59.912899 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:46:59.913233 27383 external_mini_cluster.cc:1428] Setting key 4da840e589471c50dc3c13c499d9d821
I20260807 22:46:59.918368 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.914573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:36913 (local address 127.0.0.1:46609)
0807 22:46:59.914951 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:59.914969 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:59.915002 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:46:59.915803 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:59.915822 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:59.916095 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:59.916365 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:59.917156 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:59.917161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:59.917173 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:59.917183 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:59.917269 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:59.917294 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:59.917405 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:59.917516 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 22:46:59.917659 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
I20260807 22:46:59.919121 27631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee: Bootstrap starting.
I20260807 22:46:59.920948 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:46:59.921382 27631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee: Neither blocks nor log segments found. Creating new log.
I20260807 22:46:59.922254 27631 log.cc:824] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee: Log is configured to *not* fsync() on all Append() calls
I20260807 22:46:59.924532 27631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee: No bootstrap required, opened a new log
I20260807 22:46:59.924868 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:46:59.923264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:57585 (local address 127.0.0.1:46609)
0807 22:46:59.923374 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:46:59.923378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:46:59.923616 (+ 238us) server_negotiation.cc:407] Connection header received
0807 22:46:59.923733 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:46:59.923735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:46:59.923764 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:46:59.923824 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:46:59.924460 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:46:59.924463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:46:59.924464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:46:59.924474 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:46:59.924496 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:46:59.924536 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:46:59.924610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:46:59.924721 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 22:46:59.924755 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:46:59.928035 27631 raft_consensus.cc:348] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260807 22:46:59.928246 27631 raft_consensus.cc:374] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:46:59.928289 27631 raft_consensus.cc:729] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a41d4b32cc814f1ba33a9ade4fceefee, State: Initialized, Role: FOLLOWER
I20260807 22:46:59.928738 27631 consensus_queue.cc:260] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260807 22:46:59.928874 27631 raft_consensus.cc:388] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:46:59.928936 27631 raft_consensus.cc:482] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:46:59.929028 27631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:46:59.929855 27631 raft_consensus.cc:504] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260807 22:46:59.930160 27631 leader_election.cc:302] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a41d4b32cc814f1ba33a9ade4fceefee; no voters:
I20260807 22:46:59.930442 27631 leader_election.cc:288] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:46:59.930552 27647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:46:59.930758 27647 raft_consensus.cc:686] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [term 1 LEADER]: Becoming Leader. State: Replica: a41d4b32cc814f1ba33a9ade4fceefee, State: Running, Role: LEADER
I20260807 22:46:59.931092 27647 consensus_queue.cc:237] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260807 22:46:59.931488 27631 sys_catalog.cc:564] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:46:59.932617 27649 sys_catalog.cc:455] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [sys.catalog]: SysCatalogTable state changed. Reason: New leader a41d4b32cc814f1ba33a9ade4fceefee. Latest consensus state: current_term: 1 leader_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } } }
I20260807 22:46:59.932741 27649 sys_catalog.cc:458] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:59.933049 27648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a41d4b32cc814f1ba33a9ade4fceefee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } } }
I20260807 22:46:59.933145 27648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee [sys.catalog]: This master's current role is: LEADER
I20260807 22:46:59.933141 27660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:46:59.936872 27660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:46:59.942286 27660 catalog_manager.cc:1347] Generated new cluster ID: 716d6af178c54aaca0881cf320b44c1c
I20260807 22:46:59.942379 27660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:46:59.956586 27660 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:46:59.956898 27660 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:46:59.963016 27660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a41d4b32cc814f1ba33a9ade4fceefee: Generated new TSK 0
I20260807 22:46:59.963614 27660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:46:59.967685 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27533
2026-08-07T22:46:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:00 dist-test-slave-4tdj krb5kdc[27676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:00 dist-test-slave-4tdj krb5kdc[27676](info): set up 2 sockets
Aug 07 22:47:00 dist-test-slave-4tdj krb5kdc[27676](info): commencing operation
krb5kdc: starting...
W20260807 22:47:01.111402 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:01 dist-test-slave-4tdj krb5kdc[27676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:47:01.229290 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:01.338884 27697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:01.339151 27697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:01.339191 27697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:01.343165 27697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:01.343228 27697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:01.343246 27697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:01.343262 27697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:01.343276 27697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:01.348501 27697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:01.349520 27697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:01.350528 27697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:01.358656 27704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:01.358686 27705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:01.358747 27709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:01.358829 27697 server_base.cc:1034] running on GCE node
I20260807 22:47:01.359405 27697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:01.360435 27697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:01.361611 27697 hybrid_clock.cc:648] HybridClock initialized: now 1786142821361585 us; error 37 us; skew 500 ppm
Aug 07 22:47:01 dist-test-slave-4tdj krb5kdc[27676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142821, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:01.364293 27697 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:47:01.365118 27697 webserver.cc:460] Webserver started at http://127.0.0.1:39551/ using document root <none> and password file <none>
I20260807 22:47:01.365618 27697 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:01.365676 27697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:01.365832 27697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:01.367776 27697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "d4479c7a992b440abaccccd64f7862c6"
format_stamp: "Formatted at 2026-08-07 22:47:01 on dist-test-slave-4tdj"
server_key: "4d058c9002064feafe04d64695b15aab"
server_key_iv: "8a3892cb2462a8f1c032b546c474139a"
server_key_version: "encryptionkey@0"
I20260807 22:47:01.368284 27697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "d4479c7a992b440abaccccd64f7862c6"
format_stamp: "Formatted at 2026-08-07 22:47:01 on dist-test-slave-4tdj"
server_key: "4d058c9002064feafe04d64695b15aab"
server_key_iv: "8a3892cb2462a8f1c032b546c474139a"
server_key_version: "encryptionkey@0"
I20260807 22:47:01.371950 27697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 22:47:01.374228 27720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:01.375507 27697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:47:01.375617 27697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "d4479c7a992b440abaccccd64f7862c6"
format_stamp: "Formatted at 2026-08-07 22:47:01 on dist-test-slave-4tdj"
server_key: "4d058c9002064feafe04d64695b15aab"
server_key_iv: "8a3892cb2462a8f1c032b546c474139a"
server_key_version: "encryptionkey@0"
I20260807 22:47:01.375722 27697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:01.385187 27697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:01.388203 27697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:01.388370 27697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:01.395004 27697 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32889
I20260807 22:47:01.395044 27803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32889 every 8 connection(s)
I20260807 22:47:01.395922 27697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:01.398643 27804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:01.404122 27804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6: Bootstrap starting.
I20260807 22:47:01.405342 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27697
I20260807 22:47:01.405490 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:01.405735 27383 external_mini_cluster.cc:1428] Setting key 672fa6ba282c65c0d42efc6cbf9b7081
I20260807 22:47:01.406821 27804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:01.407711 27804 log.cc:824] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:01.409816 27804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6: No bootstrap required, opened a new log
I20260807 22:47:01.413407 27804 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4479c7a992b440abaccccd64f7862c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32889 } }
I20260807 22:47:01.413666 27804 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:01.413722 27804 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4479c7a992b440abaccccd64f7862c6, State: Initialized, Role: FOLLOWER
I20260807 22:47:01.414156 27804 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4479c7a992b440abaccccd64f7862c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32889 } }
I20260807 22:47:01.414305 27804 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:01.414376 27804 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:01.414474 27804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 22:47:01 dist-test-slave-4tdj krb5kdc[27676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:47:01.415604 27804 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4479c7a992b440abaccccd64f7862c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32889 } }
I20260807 22:47:01.415930 27804 leader_election.cc:302] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4479c7a992b440abaccccd64f7862c6; no voters:
I20260807 22:47:01.416289 27804 leader_election.cc:288] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:01.416368 27815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:01.416615 27815 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [term 1 LEADER]: Becoming Leader. State: Replica: d4479c7a992b440abaccccd64f7862c6, State: Running, Role: LEADER
I20260807 22:47:01.416961 27815 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4479c7a992b440abaccccd64f7862c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32889 } }
I20260807 22:47:01.417635 27804 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:01.420284 27819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4479c7a992b440abaccccd64f7862c6. Latest consensus state: current_term: 1 leader_uuid: "d4479c7a992b440abaccccd64f7862c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4479c7a992b440abaccccd64f7862c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32889 } } }
I20260807 22:47:01.420694 27819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:01.420648 27817 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4479c7a992b440abaccccd64f7862c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4479c7a992b440abaccccd64f7862c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32889 } } }
I20260807 22:47:01.420763 27817 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:01.421649 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:01.406773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:36529 (local address 127.0.0.1:32889)
0807 22:47:01.407221 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:01.407240 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:01.407281 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:47:01.408126 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:01.408147 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:01.408486 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:01.408843 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:01.409518 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:01.410964 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:01.411820 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:01.413210 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:01.413365 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:01.416110 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:01.416117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:01.416133 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:01.416167 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:01.418160 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:01.418849 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:01.418855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:01.418861 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:01.419004 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:01.419418 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:01.419421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:01.419423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:01.419957 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:01.420189 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:01.420443 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 22:47:01.420686 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":134,"threads_started":1}
W20260807 22:47:01.423564 27841 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:01.423693 27841 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:01.423765 27842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:01.424650 27842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:01.424686 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:01.430514 27842 catalog_manager.cc:1347] Generated new cluster ID: 70f88a6c8577458cb3a9c44efd3f532e
I20260807 22:47:01.430577 27842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:01.435074 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:01.426864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:37625 (local address 127.0.0.1:32889)
0807 22:47:01.426968 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:01.426972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:01.427188 (+ 216us) server_negotiation.cc:407] Connection header received
0807 22:47:01.427305 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:01.427308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:01.427352 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:01.427408 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:01.428031 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:01.428581 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:01.429694 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:01.430035 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:01.430150 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:01.431713 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:01.431717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:01.431719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:01.431742 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:01.433231 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:01.433733 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:01.433736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:01.433738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:01.433812 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:01.434107 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:01.434110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:01.434111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:01.434321 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:01.434449 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:01.434828 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 22:47:01.434919 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:47:01.440632 27842 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:01.441807 27842 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:01.455559 27842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4479c7a992b440abaccccd64f7862c6: Generated new TSK 0
I20260807 22:47:01.456391 27842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:01.462945 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27697
W20260807 22:47:01.477504 27673 connection.cc:535] client connection to 127.0.0.1:32889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32889): Connection reset by peer (error 104)
2026-08-07T22:47:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:01 dist-test-slave-4tdj krb5kdc[27857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:01 dist-test-slave-4tdj krb5kdc[27857](info): set up 2 sockets
Aug 07 22:47:01 dist-test-slave-4tdj krb5kdc[27857](info): commencing operation
krb5kdc: starting...
W20260807 22:47:02.591730 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:02 dist-test-slave-4tdj krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:47:02.726924 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:02.832621 27878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:02.832852 27878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:02.832890 27878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:02.836745 27878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:02.836795 27878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:02.836813 27878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:02.836833 27878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:02.836848 27878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:02.841794 27878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:02.842742 27878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:02.843660 27878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:02.851362 27889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:02.851388 27893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:02.851388 27888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:02.851699 27878 server_base.cc:1034] running on GCE node
I20260807 22:47:02.852198 27878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:02.853137 27878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:02.854313 27878 hybrid_clock.cc:648] HybridClock initialized: now 1786142822854273 us; error 29 us; skew 500 ppm
Aug 07 22:47:02 dist-test-slave-4tdj krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:02.856818 27878 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:47:02.857599 27878 webserver.cc:460] Webserver started at http://127.0.0.1:35131/ using document root <none> and password file <none>
I20260807 22:47:02.858084 27878 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:02.858143 27878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:02.858354 27878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:02.860247 27878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "27697690fb474360a630b0f746f06cb5"
format_stamp: "Formatted at 2026-08-07 22:47:02 on dist-test-slave-4tdj"
server_key: "9fc6b203c311bf05cd57956ab3f455fd"
server_key_iv: "96162ffd3671b7445190eea9d1311e90"
server_key_version: "encryptionkey@0"
I20260807 22:47:02.860776 27878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "27697690fb474360a630b0f746f06cb5"
format_stamp: "Formatted at 2026-08-07 22:47:02 on dist-test-slave-4tdj"
server_key: "9fc6b203c311bf05cd57956ab3f455fd"
server_key_iv: "96162ffd3671b7445190eea9d1311e90"
server_key_version: "encryptionkey@0"
I20260807 22:47:02.864288 27878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:47:02.866525 27908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:02.867625 27878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:47:02.867738 27878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "27697690fb474360a630b0f746f06cb5"
format_stamp: "Formatted at 2026-08-07 22:47:02 on dist-test-slave-4tdj"
server_key: "9fc6b203c311bf05cd57956ab3f455fd"
server_key_iv: "96162ffd3671b7445190eea9d1311e90"
server_key_version: "encryptionkey@0"
I20260807 22:47:02.867826 27878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:02.889789 27878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:02.892884 27878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:02.893067 27878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:02.899636 27878 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35907
I20260807 22:47:02.899685 27994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35907 every 8 connection(s)
I20260807 22:47:02.900543 27878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:02.902055 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 27878
I20260807 22:47:02.902196 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:02.902499 27383 external_mini_cluster.cc:1428] Setting key b5ec9829e93b952fe77dbf4099de7fd7
I20260807 22:47:02.903486 27995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:02.909953 27995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5: Bootstrap starting.
I20260807 22:47:02.912149 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:02.903837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:42835 (local address 127.0.0.1:35907)
0807 22:47:02.904251 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:02.904269 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:02.904312 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:47:02.905112 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:02.905134 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:02.905548 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:02.905900 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:02.906349 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:02.907756 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:02.908628 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:02.909303 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:02.909453 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:02.910209 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:02.910215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:02.910230 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:02.910710 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:02.910772 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:02.910803 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:02.910933 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:02.911136 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 22:47:02.911266 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260807 22:47:02.912523 27995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:02.913195 27995 log.cc:824] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:02.914836 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:02.915115 27995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5: No bootstrap required, opened a new log
I20260807 22:47:02.919004 27995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27697690fb474360a630b0f746f06cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35907 } }
I20260807 22:47:02.919230 27995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:02.919286 27995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27697690fb474360a630b0f746f06cb5, State: Initialized, Role: FOLLOWER
I20260807 22:47:02.919694 27995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27697690fb474360a630b0f746f06cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35907 } }
I20260807 22:47:02.919857 27995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:02.919921 27995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:02.920020 27995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:02.920544 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:02.916666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:55993 (local address 127.0.0.1:35907)
0807 22:47:02.916780 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:02.916784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:02.916968 (+ 184us) server_negotiation.cc:407] Connection header received
0807 22:47:02.917092 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:02.917094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:02.917127 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:02.917186 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:02.917445 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:02.917897 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:02.918982 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:02.919202 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:02.919286 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:02.920155 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:02.920159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:02.920162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:02.920173 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:02.920200 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:02.920212 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:02.920306 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:02.920391 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:47:02.920423 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:47:02.921090 27995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27697690fb474360a630b0f746f06cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35907 } }
I20260807 22:47:02.921427 27995 leader_election.cc:302] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27697690fb474360a630b0f746f06cb5; no voters:
I20260807 22:47:02.921721 27995 leader_election.cc:288] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:02.921802 28007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:02.921990 28007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 27697690fb474360a630b0f746f06cb5, State: Running, Role: LEADER
I20260807 22:47:02.922286 28007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27697690fb474360a630b0f746f06cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35907 } }
I20260807 22:47:02.922874 27995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:02.923856 28009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27697690fb474360a630b0f746f06cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27697690fb474360a630b0f746f06cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35907 } } }
I20260807 22:47:02.923995 28009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:02.924250 28010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27697690fb474360a630b0f746f06cb5. Latest consensus state: current_term: 1 leader_uuid: "27697690fb474360a630b0f746f06cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27697690fb474360a630b0f746f06cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35907 } } }
I20260807 22:47:02.924345 28010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:02.924429 28019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:02.927376 28019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:02.932494 28019 catalog_manager.cc:1347] Generated new cluster ID: e45e9dfc724d4dc09daff3ec1eb96c2d
I20260807 22:47:02.932555 28019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:02.950110 28019 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:02.951001 28019 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:02.959436 28019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27697690fb474360a630b0f746f06cb5: Generated new TSK 0
I20260807 22:47:02.960049 28019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:02.969349 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 27878
W20260807 22:47:02.983356 27852 connection.cc:535] client connection to 127.0.0.1:35907 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35907): Connection reset by peer (error 104)
2026-08-07T22:47:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:03 dist-test-slave-4tdj krb5kdc[28039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:03 dist-test-slave-4tdj krb5kdc[28039](info): set up 2 sockets
Aug 07 22:47:03 dist-test-slave-4tdj krb5kdc[28039](info): commencing operation
krb5kdc: starting...
W20260807 22:47:04.099578 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:04 dist-test-slave-4tdj krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:47:04.215353 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:04.323105 28054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:04.323426 28054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:04.323467 28054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:04.327373 28054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:04.327415 28054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:04.327433 28054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:04.327447 28054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:04.327461 28054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:04.332520 28054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33461
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:04.333443 28054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:04.334375 28054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:04.342394 28059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:04.342479 28054 server_base.cc:1034] running on GCE node
W20260807 22:47:04.342429 28061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:04.342424 28064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:04.343164 28054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:04.344208 28054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:04.345363 28054 hybrid_clock.cc:648] HybridClock initialized: now 1786142824345343 us; error 38 us; skew 500 ppm
Aug 07 22:47:04 dist-test-slave-4tdj krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:04.347999 28054 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:47:04.348800 28054 webserver.cc:460] Webserver started at http://127.0.0.1:43313/ using document root <none> and password file <none>
I20260807 22:47:04.349278 28054 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:04.349332 28054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:04.349488 28054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:04.351425 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "2e59c3d2f8474b299fa79fb92bc7c735"
format_stamp: "Formatted at 2026-08-07 22:47:04 on dist-test-slave-4tdj"
server_key: "40141bdda5831140c72d72b179a0dfac"
server_key_iv: "71f5944a1ba7a37046a4a219825bb40f"
server_key_version: "encryptionkey@0"
I20260807 22:47:04.351933 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "2e59c3d2f8474b299fa79fb92bc7c735"
format_stamp: "Formatted at 2026-08-07 22:47:04 on dist-test-slave-4tdj"
server_key: "40141bdda5831140c72d72b179a0dfac"
server_key_iv: "71f5944a1ba7a37046a4a219825bb40f"
server_key_version: "encryptionkey@0"
I20260807 22:47:04.355350 28054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 22:47:04.357540 28073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:04.358724 28054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 22:47:04.358834 28054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "2e59c3d2f8474b299fa79fb92bc7c735"
format_stamp: "Formatted at 2026-08-07 22:47:04 on dist-test-slave-4tdj"
server_key: "40141bdda5831140c72d72b179a0dfac"
server_key_iv: "71f5944a1ba7a37046a4a219825bb40f"
server_key_version: "encryptionkey@0"
I20260807 22:47:04.358920 28054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:04.363219 28054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:04.365911 28054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:04.366065 28054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:04.372717 28054 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33461
I20260807 22:47:04.372757 28161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33461 every 8 connection(s)
I20260807 22:47:04.373543 28054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:04.376261 28162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:04.380698 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28054
I20260807 22:47:04.380841 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:04.381094 27383 external_mini_cluster.cc:1428] Setting key 6a3e31f78fa93b6aed07589b538af586
I20260807 22:47:04.381899 28162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735: Bootstrap starting.
I20260807 22:47:04.384691 28162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:04.385685 28162 log.cc:824] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:04.386441 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:04.382337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:60973 (local address 127.0.0.1:33461)
0807 22:47:04.382779 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:04.382798 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:04.382841 (+ 43us) server_negotiation.cc:407] Connection header received
0807 22:47:04.383679 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:04.383703 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:04.384047 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:04.384396 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:04.385050 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:04.385057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:04.385071 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:04.385083 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:04.385142 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:04.385168 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:04.385293 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:04.385417 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:47:04.385574 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":100,"threads_started":1}
I20260807 22:47:04.387890 28162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735: No bootstrap required, opened a new log
I20260807 22:47:04.389117 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:04.391773 28162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260807 22:47:04.391994 28162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:04.392056 28162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e59c3d2f8474b299fa79fb92bc7c735, State: Initialized, Role: FOLLOWER
I20260807 22:47:04.392531 28162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260807 22:47:04.392647 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:04.391076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:37633 (local address 127.0.0.1:33461)
0807 22:47:04.391176 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:04.391181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:04.391368 (+ 187us) server_negotiation.cc:407] Connection header received
0807 22:47:04.391444 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:04.391447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:04.391482 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:04.391573 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:04.392230 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:04.392234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:04.392237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:04.392252 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:04.392283 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:04.392297 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:04.392393 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:04.392454 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 22:47:04.392493 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 22:47:04.392706 28162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:04.392778 28162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:04.392885 28162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:04.393864 28162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260807 22:47:04.394161 28162 leader_election.cc:302] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e59c3d2f8474b299fa79fb92bc7c735; no voters:
I20260807 22:47:04.394474 28162 leader_election.cc:288] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:04.394536 28172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:04.394742 28172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [term 1 LEADER]: Becoming Leader. State: Replica: 2e59c3d2f8474b299fa79fb92bc7c735, State: Running, Role: LEADER
I20260807 22:47:04.395094 28172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260807 22:47:04.395623 28162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:04.396742 28175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e59c3d2f8474b299fa79fb92bc7c735. Latest consensus state: current_term: 1 leader_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } } }
I20260807 22:47:04.396751 28173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e59c3d2f8474b299fa79fb92bc7c735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } } }
I20260807 22:47:04.396895 28175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:04.396896 28173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:04.397320 28184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:04.400116 28184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:04.405109 28184 catalog_manager.cc:1347] Generated new cluster ID: 2b2bbe3d942740f895b9ef4b4ec17ba0
I20260807 22:47:04.405237 28184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:04.419003 28184 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:04.419294 28184 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:04.424487 28184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e59c3d2f8474b299fa79fb92bc7c735: Generated new TSK 0
I20260807 22:47:04.425118 28184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:04.436378 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28054
2026-08-07T22:47:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:04 dist-test-slave-4tdj krb5kdc[28196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:04 dist-test-slave-4tdj krb5kdc[28196](info): set up 2 sockets
Aug 07 22:47:04 dist-test-slave-4tdj krb5kdc[28196](info): commencing operation
krb5kdc: starting...
W20260807 22:47:05.571774 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:05 dist-test-slave-4tdj krb5kdc[28196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:05.688876 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43225
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:05.797622 28215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:05.797864 28215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:05.797904 28215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:05.801815 28215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:05.801872 28215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:05.801889 28215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:05.801923 28215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:05.801950 28215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:05.807088 28215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:05.808063 28215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:05.809037 28215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:05.817307 28227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:05.817309 28226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:05.817330 28229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:05.817449 28215 server_base.cc:1034] running on GCE node
I20260807 22:47:05.818058 28215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:05.819129 28215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:05.820298 28215 hybrid_clock.cc:648] HybridClock initialized: now 1786142825820283 us; error 25 us; skew 500 ppm
Aug 07 22:47:05 dist-test-slave-4tdj krb5kdc[28196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142825, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:05.825839 28215 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:05.826783 28215 webserver.cc:460] Webserver started at http://127.26.189.254:40497/ using document root <none> and password file <none>
I20260807 22:47:05.827301 28215 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:05.827368 28215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:05.827579 28215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:05.829571 28215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "b0f5e99b99d243abadd975680dcc88b9"
format_stamp: "Formatted at 2026-08-07 22:47:05 on dist-test-slave-4tdj"
server_key: "bdbbd14e6d2299c8c685ef9dce13eaa1"
server_key_iv: "0af03e26018a683374c956c7dad543ff"
server_key_version: "encryptionkey@0"
I20260807 22:47:05.830104 28215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "b0f5e99b99d243abadd975680dcc88b9"
format_stamp: "Formatted at 2026-08-07 22:47:05 on dist-test-slave-4tdj"
server_key: "bdbbd14e6d2299c8c685ef9dce13eaa1"
server_key_iv: "0af03e26018a683374c956c7dad543ff"
server_key_version: "encryptionkey@0"
I20260807 22:47:05.833670 28215 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:47:05.836107 28240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:05.837311 28215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:47:05.837423 28215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "b0f5e99b99d243abadd975680dcc88b9"
format_stamp: "Formatted at 2026-08-07 22:47:05 on dist-test-slave-4tdj"
server_key: "bdbbd14e6d2299c8c685ef9dce13eaa1"
server_key_iv: "0af03e26018a683374c956c7dad543ff"
server_key_version: "encryptionkey@0"
I20260807 22:47:05.837512 28215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:05.859215 28215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:05.861943 28215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:05.862118 28215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:05.869021 28215 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43225
I20260807 22:47:05.869041 28323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43225 every 8 connection(s)
I20260807 22:47:05.869957 28215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:05.872622 28324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:05.874853 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28215
I20260807 22:47:05.874984 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:05.875226 27383 external_mini_cluster.cc:1428] Setting key 9791fb644708b3e2ecafc5b7e439c08b
I20260807 22:47:05.878439 28324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9: Bootstrap starting.
I20260807 22:47:05.880636 28324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:05.881362 28324 log.cc:824] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:05.883267 28324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9: No bootstrap required, opened a new log
Aug 07 22:47:05 dist-test-slave-4tdj krb5kdc[28196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:05.886866 28324 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f5e99b99d243abadd975680dcc88b9" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43225 } }
I20260807 22:47:05.887121 28324 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:05.887187 28324 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f5e99b99d243abadd975680dcc88b9, State: Initialized, Role: FOLLOWER
I20260807 22:47:05.887642 28324 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f5e99b99d243abadd975680dcc88b9" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43225 } }
I20260807 22:47:05.887800 28324 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:05.887871 28324 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:05.887979 28324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:05.889004 28324 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f5e99b99d243abadd975680dcc88b9" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43225 } }
I20260807 22:47:05.889326 28324 leader_election.cc:302] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0f5e99b99d243abadd975680dcc88b9; no voters:
I20260807 22:47:05.889626 28324 leader_election.cc:288] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:05.889704 28331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:05.889930 28331 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [term 1 LEADER]: Becoming Leader. State: Replica: b0f5e99b99d243abadd975680dcc88b9, State: Running, Role: LEADER
I20260807 22:47:05.890031 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:05.876541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56573 (local address 127.26.189.254:43225)
0807 22:47:05.876995 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:05.877009 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:05.877056 (+ 47us) server_negotiation.cc:407] Connection header received
0807 22:47:05.877698 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:05.877715 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:05.878042 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:05.878375 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:05.879207 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:05.880176 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:05.881111 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:05.881608 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:05.881754 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:05.884484 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:05.884489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:05.884502 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:05.884530 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:05.886875 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:05.887443 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:05.887451 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:05.887456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:05.887614 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:05.887957 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:05.887961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:05.887963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:05.888396 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:05.888569 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:05.888894 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:47:05.889101 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":138,"threads_started":1}
I20260807 22:47:05.890331 28331 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f5e99b99d243abadd975680dcc88b9" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43225 } }
I20260807 22:47:05.890870 28324 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:05.892518 28334 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0f5e99b99d243abadd975680dcc88b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f5e99b99d243abadd975680dcc88b9" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43225 } } }
I20260807 22:47:05.892658 28334 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:05.893474 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:05.893312 28336 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0f5e99b99d243abadd975680dcc88b9. Latest consensus state: current_term: 1 leader_uuid: "b0f5e99b99d243abadd975680dcc88b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0f5e99b99d243abadd975680dcc88b9" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43225 } } }
I20260807 22:47:05.893445 28353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:05.893678 28336 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:05.896703 28353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:05.902073 28353 catalog_manager.cc:1347] Generated new cluster ID: 8277abd6fe264810a2cc426785d5d31f
I20260807 22:47:05.902138 28353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:05.908742 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:05.895660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43377 (local address 127.26.189.254:43225)
0807 22:47:05.895758 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:05.895762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:05.896003 (+ 241us) server_negotiation.cc:407] Connection header received
0807 22:47:05.896128 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:05.896131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:05.896175 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:05.896256 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:05.897035 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:05.897787 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:05.898951 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:05.899291 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:05.899394 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:05.901007 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:05.901010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:05.901013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:05.901039 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:05.907035 (+ 5996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:05.907506 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:05.907510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:05.907512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:05.907593 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:05.907912 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:05.907916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:05.907918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:05.908094 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:05.908247 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:05.908488 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 22:47:05.908579 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:47:05.912119 28353 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:05.912923 28353 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:05.929158 28353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0f5e99b99d243abadd975680dcc88b9: Generated new TSK 0
I20260807 22:47:05.929806 28353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:05.935297 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28215
W20260807 22:47:05.951654 28194 connection.cc:535] client connection to 127.26.189.254:43225 recv error: Network error: failed to read from TLS socket (remote: 127.26.189.254:43225): Connection reset by peer (error 104)
2026-08-07T22:47:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:05 dist-test-slave-4tdj krb5kdc[28371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:06 dist-test-slave-4tdj krb5kdc[28371](info): set up 2 sockets
Aug 07 22:47:06 dist-test-slave-4tdj krb5kdc[28371](info): commencing operation
krb5kdc: starting...
W20260807 22:47:07.079480 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:07 dist-test-slave-4tdj krb5kdc[28371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:07.199290 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43113
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:43113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:07.307015 28388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:07.307250 28388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:07.307307 28388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:07.311193 28388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:07.311250 28388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:07.311275 28388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:07.311297 28388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:07.311326 28388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:07.316442 28388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:43113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:43113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:07.317410 28388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:07.318420 28388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:07.326565 28395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:07.326694 28398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:07.326643 28400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:07.326861 28388 server_base.cc:1034] running on GCE node
I20260807 22:47:07.327457 28388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:07.328509 28388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:07.329674 28388 hybrid_clock.cc:648] HybridClock initialized: now 1786142827329659 us; error 26 us; skew 500 ppm
Aug 07 22:47:07 dist-test-slave-4tdj krb5kdc[28371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142827, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:07.332324 28388 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:07.333168 28388 webserver.cc:460] Webserver started at http://127.26.189.254:35451/ using document root <none> and password file <none>
I20260807 22:47:07.333681 28388 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:07.333747 28388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:07.333943 28388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:07.335950 28388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "76447af1d92e4b629f3dcade971047a8"
format_stamp: "Formatted at 2026-08-07 22:47:07 on dist-test-slave-4tdj"
server_key: "ea3316514240e4f0da6d497d978109da"
server_key_iv: "261c39d1b890f4ca39545c170db9b202"
server_key_version: "encryptionkey@0"
I20260807 22:47:07.336491 28388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "76447af1d92e4b629f3dcade971047a8"
format_stamp: "Formatted at 2026-08-07 22:47:07 on dist-test-slave-4tdj"
server_key: "ea3316514240e4f0da6d497d978109da"
server_key_iv: "261c39d1b890f4ca39545c170db9b202"
server_key_version: "encryptionkey@0"
I20260807 22:47:07.340019 28388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:47:07.342253 28414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:07.343452 28388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:47:07.343576 28388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "76447af1d92e4b629f3dcade971047a8"
format_stamp: "Formatted at 2026-08-07 22:47:07 on dist-test-slave-4tdj"
server_key: "ea3316514240e4f0da6d497d978109da"
server_key_iv: "261c39d1b890f4ca39545c170db9b202"
server_key_version: "encryptionkey@0"
I20260807 22:47:07.343667 28388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:07.363823 28388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:07.366745 28388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:07.366925 28388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:07.374027 28388 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:43113
I20260807 22:47:07.374042 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:43113 every 8 connection(s)
I20260807 22:47:07.374958 28388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:07.377566 28494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:07.382850 28494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8: Bootstrap starting.
I20260807 22:47:07.384856 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28388
I20260807 22:47:07.385004 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:07.385327 27383 external_mini_cluster.cc:1428] Setting key c0193c7b686acedaf0476357bdab23f0
I20260807 22:47:07.385370 28494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:07.386178 28494 log.cc:824] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:07.388404 28494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8: No bootstrap required, opened a new log
I20260807 22:47:07.392521 28494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76447af1d92e4b629f3dcade971047a8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43113 } }
I20260807 22:47:07.392771 28494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:07.392839 28494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76447af1d92e4b629f3dcade971047a8, State: Initialized, Role: FOLLOWER
I20260807 22:47:07.393287 28494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76447af1d92e4b629f3dcade971047a8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43113 } }
I20260807 22:47:07.393445 28494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:07.393514 28494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:07.393625 28494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:07.394518 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:07.386462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57137 (local address 127.26.189.254:43113)
0807 22:47:07.386815 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:07.386829 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:07.386862 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:47:07.387638 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:07.387660 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:07.388064 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:07.388402 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:07.388807 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:07.390276 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:07.391158 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:07.391844 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:07.391999 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:07.392724 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:07.392731 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:07.392747 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:07.393229 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:07.393292 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:07.393321 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:07.393454 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:07.393677 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 22:47:07.393801 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
I20260807 22:47:07.394541 28494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76447af1d92e4b629f3dcade971047a8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43113 } }
I20260807 22:47:07.394850 28494 leader_election.cc:302] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76447af1d92e4b629f3dcade971047a8; no voters:
I20260807 22:47:07.395116 28494 leader_election.cc:288] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:07.395218 28501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:07.395488 28501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [term 1 LEADER]: Becoming Leader. State: Replica: 76447af1d92e4b629f3dcade971047a8, State: Running, Role: LEADER
I20260807 22:47:07.395818 28501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76447af1d92e4b629f3dcade971047a8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43113 } }
I20260807 22:47:07.396206 28494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:07.397352 28502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76447af1d92e4b629f3dcade971047a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76447af1d92e4b629f3dcade971047a8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43113 } } }
I20260807 22:47:07.397475 28502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:07.397650 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:07.397802 28504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76447af1d92e4b629f3dcade971047a8. Latest consensus state: current_term: 1 leader_uuid: "76447af1d92e4b629f3dcade971047a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76447af1d92e4b629f3dcade971047a8" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 43113 } } }
I20260807 22:47:07.397881 28512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:07.397886 28504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:07.400967 28512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:07.404577 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:07.400030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45863 (local address 127.26.189.254:43113)
0807 22:47:07.400135 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:07.400139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:07.400438 (+ 299us) server_negotiation.cc:407] Connection header received
0807 22:47:07.400537 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:07.400540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:07.400589 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:07.400664 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:07.400982 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:07.401691 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:07.402848 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:07.403157 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:07.403260 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:07.404127 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:07.404131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:07.404134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:07.404147 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:07.404174 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:07.404187 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:07.404270 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:07.404415 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 22:47:07.404449 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:47:07.406591 28512 catalog_manager.cc:1347] Generated new cluster ID: de1cb5b55d6948bea8cb2548f8e99b0f
I20260807 22:47:07.406647 28512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:07.415496 28512 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:07.416293 28512 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:07.422785 28512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76447af1d92e4b629f3dcade971047a8: Generated new TSK 0
I20260807 22:47:07.423348 28512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:07.426803 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28388
W20260807 22:47:07.439903 28367 connection.cc:535] client connection to 127.26.189.254:43113 recv error: Network error: failed to read from TLS socket (remote: 127.26.189.254:43113): Connection reset by peer (error 104)
2026-08-07T22:47:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:07 dist-test-slave-4tdj krb5kdc[28539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:07 dist-test-slave-4tdj krb5kdc[28539](info): set up 2 sockets
Aug 07 22:47:07 dist-test-slave-4tdj krb5kdc[28539](info): commencing operation
krb5kdc: starting...
W20260807 22:47:08.553915 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:08 dist-test-slave-4tdj krb5kdc[28539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:08.675407 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:33585
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:33585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:08.782661 28561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:08.782912 28561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:08.782971 28561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:08.786898 28561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:08.786957 28561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:08.786983 28561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:08.787019 28561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:08.787050 28561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:08.792124 28561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:33585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:33585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:08.793115 28561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:08.794063 28561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:47:08.802122 28561 server_base.cc:1034] running on GCE node
W20260807 22:47:08.802090 28567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:08.802076 28573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:08.802088 28569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:08.802843 28561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:08.803889 28561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:08.805047 28561 hybrid_clock.cc:648] HybridClock initialized: now 1786142828805027 us; error 36 us; skew 500 ppm
Aug 07 22:47:08 dist-test-slave-4tdj krb5kdc[28539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142828, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:08.807662 28561 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:08.808445 28561 webserver.cc:460] Webserver started at http://127.26.189.254:36089/ using document root <none> and password file <none>
I20260807 22:47:08.808921 28561 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:08.808976 28561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:08.809141 28561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:08.811060 28561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "3425bd731428483e9cc8020d6859252c"
format_stamp: "Formatted at 2026-08-07 22:47:08 on dist-test-slave-4tdj"
server_key: "89c8cfb4706694db5759b03f2d0cade2"
server_key_iv: "58d1c8550921114b0648db5705979608"
server_key_version: "encryptionkey@0"
I20260807 22:47:08.811563 28561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "3425bd731428483e9cc8020d6859252c"
format_stamp: "Formatted at 2026-08-07 22:47:08 on dist-test-slave-4tdj"
server_key: "89c8cfb4706694db5759b03f2d0cade2"
server_key_iv: "58d1c8550921114b0648db5705979608"
server_key_version: "encryptionkey@0"
I20260807 22:47:08.815013 28561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:47:08.817255 28581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:08.818446 28561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:47:08.818562 28561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "3425bd731428483e9cc8020d6859252c"
format_stamp: "Formatted at 2026-08-07 22:47:08 on dist-test-slave-4tdj"
server_key: "89c8cfb4706694db5759b03f2d0cade2"
server_key_iv: "58d1c8550921114b0648db5705979608"
server_key_version: "encryptionkey@0"
I20260807 22:47:08.818670 28561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:08.822991 28561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:08.825887 28561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:08.826043 28561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:08.832799 28561 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:33585
I20260807 22:47:08.832808 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:33585 every 8 connection(s)
I20260807 22:47:08.833696 28561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:08.836289 28659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:08.840895 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28561
I20260807 22:47:08.841020 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:08.841270 27383 external_mini_cluster.cc:1428] Setting key a3e2e59e5a4cbef17d739a15072687c8
I20260807 22:47:08.841610 28659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c: Bootstrap starting.
I20260807 22:47:08.844198 28659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:08.845070 28659 log.cc:824] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:08.846599 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:08.842597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59501 (local address 127.26.189.254:33585)
0807 22:47:08.843027 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:08.843046 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:08.843088 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:47:08.843892 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:08.843915 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:08.844262 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:08.844633 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:08.845326 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:08.845332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:08.845345 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:08.845356 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:08.845413 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:08.845432 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:08.845526 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:08.845687 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 22:47:08.845847 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":111,"threads_started":1}
I20260807 22:47:08.846932 28659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c: No bootstrap required, opened a new log
I20260807 22:47:08.849308 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:08.851164 28659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3425bd731428483e9cc8020d6859252c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33585 } }
I20260807 22:47:08.851418 28659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:08.851517 28659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3425bd731428483e9cc8020d6859252c, State: Initialized, Role: FOLLOWER
I20260807 22:47:08.852025 28659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3425bd731428483e9cc8020d6859252c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33585 } }
I20260807 22:47:08.852209 28659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:08.852283 28659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:08.852384 28659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:08.853111 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:08.851550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34743 (local address 127.26.189.254:33585)
0807 22:47:08.851670 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:08.851674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:08.851901 (+ 227us) server_negotiation.cc:407] Connection header received
0807 22:47:08.852008 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:08.852010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:08.852038 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:08.852108 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:08.852787 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:08.852790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:08.852792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:08.852802 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:08.852826 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:08.852836 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:08.852900 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:08.852971 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 22:47:08.852998 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:47:08.853546 28659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3425bd731428483e9cc8020d6859252c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33585 } }
I20260807 22:47:08.853914 28659 leader_election.cc:302] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3425bd731428483e9cc8020d6859252c; no voters:
I20260807 22:47:08.854219 28659 leader_election.cc:288] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:08.854337 28676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:08.854544 28676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [term 1 LEADER]: Becoming Leader. State: Replica: 3425bd731428483e9cc8020d6859252c, State: Running, Role: LEADER
I20260807 22:47:08.854887 28676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3425bd731428483e9cc8020d6859252c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33585 } }
I20260807 22:47:08.855319 28659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:08.856328 28678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3425bd731428483e9cc8020d6859252c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3425bd731428483e9cc8020d6859252c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33585 } } }
I20260807 22:47:08.856608 28679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3425bd731428483e9cc8020d6859252c. Latest consensus state: current_term: 1 leader_uuid: "3425bd731428483e9cc8020d6859252c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3425bd731428483e9cc8020d6859252c" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 33585 } } }
I20260807 22:47:08.856716 28678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:08.856791 28679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c [sys.catalog]: This master's current role is: LEADER
W20260807 22:47:08.859845 28698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:08.859910 28698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:08.860019 28694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:08.860894 28694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:08.866137 28694 catalog_manager.cc:1347] Generated new cluster ID: 9a632ca903174def8b21243eb18a7229
I20260807 22:47:08.866196 28694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:08.905134 28694 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:08.905462 28694 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:08.915541 28694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3425bd731428483e9cc8020d6859252c: Generated new TSK 0
I20260807 22:47:08.916193 28694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:08.919133 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28561
2026-08-07T22:47:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:08 dist-test-slave-4tdj krb5kdc[28707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:08 dist-test-slave-4tdj krb5kdc[28707](info): set up 2 sockets
Aug 07 22:47:08 dist-test-slave-4tdj krb5kdc[28707](info): commencing operation
krb5kdc: starting...
W20260807 22:47:10.056311 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:10 dist-test-slave-4tdj krb5kdc[28707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:10.176939 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38329
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:10.285384 28726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:10.285650 28726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:10.285707 28726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:10.289672 28726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:10.289729 28726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:10.289757 28726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:10.289778 28726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:10.289803 28726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:10.294929 28726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:38329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:10.295888 28726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:10.296864 28726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:10.304742 28737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:10.304795 28740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:10.304937 28738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:10.305109 28726 server_base.cc:1034] running on GCE node
I20260807 22:47:10.305611 28726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:10.306686 28726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:10.307847 28726 hybrid_clock.cc:648] HybridClock initialized: now 1786142830307834 us; error 24 us; skew 500 ppm
Aug 07 22:47:10 dist-test-slave-4tdj krb5kdc[28707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142830, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:10.310446 28726 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:10.311240 28726 webserver.cc:460] Webserver started at http://127.26.189.254:37669/ using document root <none> and password file <none>
I20260807 22:47:10.311722 28726 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:10.311775 28726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:10.311936 28726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:10.313868 28726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "014d59382bc9464dab1935540ce663fd"
format_stamp: "Formatted at 2026-08-07 22:47:10 on dist-test-slave-4tdj"
server_key: "d84303bd548069f25c3f37783d7a44f7"
server_key_iv: "ce7d4d3be68ed44e132bffe3e4033899"
server_key_version: "encryptionkey@0"
I20260807 22:47:10.314428 28726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "014d59382bc9464dab1935540ce663fd"
format_stamp: "Formatted at 2026-08-07 22:47:10 on dist-test-slave-4tdj"
server_key: "d84303bd548069f25c3f37783d7a44f7"
server_key_iv: "ce7d4d3be68ed44e132bffe3e4033899"
server_key_version: "encryptionkey@0"
I20260807 22:47:10.317829 28726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:47:10.320148 28751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:10.321368 28726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:47:10.321471 28726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "014d59382bc9464dab1935540ce663fd"
format_stamp: "Formatted at 2026-08-07 22:47:10 on dist-test-slave-4tdj"
server_key: "d84303bd548069f25c3f37783d7a44f7"
server_key_iv: "ce7d4d3be68ed44e132bffe3e4033899"
server_key_version: "encryptionkey@0"
I20260807 22:47:10.321552 28726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:10.335307 28726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:10.338212 28726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:10.338395 28726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:10.345117 28726 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:38329
I20260807 22:47:10.345124 28826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:38329 every 8 connection(s)
I20260807 22:47:10.346029 28726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:10.348733 28828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:10.352942 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28726
I20260807 22:47:10.353085 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:10.353396 27383 external_mini_cluster.cc:1428] Setting key f26929977eaa43d876151d5217506edd
I20260807 22:47:10.354411 28828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd: Bootstrap starting.
I20260807 22:47:10.356665 28828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:10.357362 28828 log.cc:824] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:10.359133 28828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd: No bootstrap required, opened a new log
Aug 07 22:47:10 dist-test-slave-4tdj krb5kdc[28707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:10.362800 28828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014d59382bc9464dab1935540ce663fd" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38329 } }
I20260807 22:47:10.363004 28828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:10.363051 28828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014d59382bc9464dab1935540ce663fd, State: Initialized, Role: FOLLOWER
I20260807 22:47:10.363461 28828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014d59382bc9464dab1935540ce663fd" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38329 } }
I20260807 22:47:10.363580 28828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:10.363631 28828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:10.363722 28828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:10.364567 28828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014d59382bc9464dab1935540ce663fd" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38329 } }
I20260807 22:47:10.364912 28828 leader_election.cc:302] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 014d59382bc9464dab1935540ce663fd; no voters:
I20260807 22:47:10.365240 28828 leader_election.cc:288] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:10.365353 28834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:10.365566 28834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [term 1 LEADER]: Becoming Leader. State: Replica: 014d59382bc9464dab1935540ce663fd, State: Running, Role: LEADER
I20260807 22:47:10.365936 28834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014d59382bc9464dab1935540ce663fd" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38329 } }
I20260807 22:47:10.366490 28828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:10.367692 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:10.354814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44523 (local address 127.26.189.254:38329)
0807 22:47:10.355235 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:10.355249 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:10.355284 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:47:10.355900 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:10.355916 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:10.356248 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:10.356538 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:10.357338 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:10.358349 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:10.359194 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:10.359668 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:10.359812 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:10.362468 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:10.362474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:10.362486 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:10.362512 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:10.364213 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:10.364680 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:10.364686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:10.364692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:10.364809 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:10.365128 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:10.365130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:10.365131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:10.365638 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:10.365830 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:10.366423 (+ 593us) server_negotiation.cc:299] Negotiation successful
0807 22:47:10.366713 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260807 22:47:10.369031 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 014d59382bc9464dab1935540ce663fd. Latest consensus state: current_term: 1 leader_uuid: "014d59382bc9464dab1935540ce663fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014d59382bc9464dab1935540ce663fd" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38329 } } }
I20260807 22:47:10.369153 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:10.369554 28857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:10.369925 28836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "014d59382bc9464dab1935540ce663fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014d59382bc9464dab1935540ce663fd" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 38329 } } }
I20260807 22:47:10.370071 28836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:10.371243 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:10.372313 28857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:10.377722 28857 catalog_manager.cc:1347] Generated new cluster ID: e64a46c0b9954dd2b99413d2add1fb83
I20260807 22:47:10.377828 28857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:10.382691 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:10.374762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59817 (local address 127.26.189.254:38329)
0807 22:47:10.374860 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:10.374864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:10.375082 (+ 218us) server_negotiation.cc:407] Connection header received
0807 22:47:10.375180 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:10.375183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:10.375225 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:10.375284 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:10.375898 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:10.376600 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:10.377476 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:10.377804 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:10.377883 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:10.379526 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:10.379529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:10.379532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:10.379566 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:10.381117 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:10.381530 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:10.381533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:10.381535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:10.381632 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:10.381861 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:10.381864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:10.381866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:10.382073 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:10.382201 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:10.382418 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 22:47:10.382534 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:47:10.401374 28857 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:10.402338 28857 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:10.415217 28857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 014d59382bc9464dab1935540ce663fd: Generated new TSK 0
I20260807 22:47:10.416061 28857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:10.428758 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28726
2026-08-07T22:47:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:10 dist-test-slave-4tdj krb5kdc[28874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:10 dist-test-slave-4tdj krb5kdc[28874](info): set up 2 sockets
Aug 07 22:47:10 dist-test-slave-4tdj krb5kdc[28874](info): commencing operation
krb5kdc: starting...
W20260807 22:47:11.561468 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:11 dist-test-slave-4tdj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:11.679153 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:35493
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:11.785852 28893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:11.786075 28893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:11.786113 28893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:11.789952 28893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:11.790004 28893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:11.790021 28893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:11.790036 28893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:11.790059 28893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:11.795094 28893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:35493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:11.795998 28893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:11.796921 28893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:11.804862 28899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:11.804885 28898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:11.804885 28902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:11.805135 28893 server_base.cc:1034] running on GCE node
I20260807 22:47:11.805605 28893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:11.806617 28893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:11.807767 28893 hybrid_clock.cc:648] HybridClock initialized: now 1786142831807756 us; error 26 us; skew 500 ppm
Aug 07 22:47:11 dist-test-slave-4tdj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142831, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:11.813378 28893 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:11.814217 28893 webserver.cc:460] Webserver started at http://127.26.189.254:35803/ using document root <none> and password file <none>
I20260807 22:47:11.814728 28893 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:11.814785 28893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:11.814945 28893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:11.816797 28893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "6e974a79fe2a44c98d6853d3f92a181e"
format_stamp: "Formatted at 2026-08-07 22:47:11 on dist-test-slave-4tdj"
server_key: "0a664e49a8e4c46420068909a92cf6d3"
server_key_iv: "ae83f431c992d14c479f2e2ef7315e6e"
server_key_version: "encryptionkey@0"
I20260807 22:47:11.817282 28893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "6e974a79fe2a44c98d6853d3f92a181e"
format_stamp: "Formatted at 2026-08-07 22:47:11 on dist-test-slave-4tdj"
server_key: "0a664e49a8e4c46420068909a92cf6d3"
server_key_iv: "ae83f431c992d14c479f2e2ef7315e6e"
server_key_version: "encryptionkey@0"
I20260807 22:47:11.820770 28893 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:47:11.822975 28912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:11.824090 28893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:47:11.824196 28893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "6e974a79fe2a44c98d6853d3f92a181e"
format_stamp: "Formatted at 2026-08-07 22:47:11 on dist-test-slave-4tdj"
server_key: "0a664e49a8e4c46420068909a92cf6d3"
server_key_iv: "ae83f431c992d14c479f2e2ef7315e6e"
server_key_version: "encryptionkey@0"
I20260807 22:47:11.824285 28893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:11.850396 28893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:11.853457 28893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:11.853610 28893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:11.860263 28893 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:35493
I20260807 22:47:11.860311 28985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:35493 every 8 connection(s)
I20260807 22:47:11.861141 28893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:11.863776 28986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:11.865001 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 28893
I20260807 22:47:11.865123 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:11.865394 27383 external_mini_cluster.cc:1428] Setting key 204c646382ceee4e0a2ca3238306dcf9
I20260807 22:47:11.869455 28986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e: Bootstrap starting.
I20260807 22:47:11.871629 28986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:11.872387 28986 log.cc:824] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:11.873917 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:11.866735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57033 (local address 127.26.189.254:35493)
0807 22:47:11.867184 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:11.867198 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:11.867234 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:47:11.867858 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:11.867875 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:11.868195 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:11.868488 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:11.868891 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:11.869852 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:11.870767 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:11.871224 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:11.871365 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:11.872162 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:11.872167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:11.872180 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:11.872558 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:11.872616 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:11.872640 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:11.872725 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:11.872856 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:47:11.873060 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":125,"threads_started":1}
I20260807 22:47:11.874485 28986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e: No bootstrap required, opened a new log
I20260807 22:47:11.876629 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:11.878072 28986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e974a79fe2a44c98d6853d3f92a181e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35493 } }
I20260807 22:47:11.878324 28986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:11.878427 28986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e974a79fe2a44c98d6853d3f92a181e, State: Initialized, Role: FOLLOWER
I20260807 22:47:11.878863 28986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e974a79fe2a44c98d6853d3f92a181e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35493 } }
I20260807 22:47:11.879024 28986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:11.879098 28986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:11.879218 28986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:11.880053 28986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e974a79fe2a44c98d6853d3f92a181e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35493 } }
I20260807 22:47:11.880349 28986 leader_election.cc:302] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e974a79fe2a44c98d6853d3f92a181e; no voters:
I20260807 22:47:11.880616 28986 leader_election.cc:288] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:11.880760 28997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:11.880999 28997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [term 1 LEADER]: Becoming Leader. State: Replica: 6e974a79fe2a44c98d6853d3f92a181e, State: Running, Role: LEADER
I20260807 22:47:11.881321 28997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e974a79fe2a44c98d6853d3f92a181e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35493 } }
I20260807 22:47:11.881840 28986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:11.882421 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:11.878508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41975 (local address 127.26.189.254:35493)
0807 22:47:11.878615 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:11.878620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:11.878808 (+ 188us) server_negotiation.cc:407] Connection header received
0807 22:47:11.878885 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:11.878888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:11.878929 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:11.879016 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:11.879381 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:11.879874 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:11.880769 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:11.881114 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:11.881218 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:11.881953 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:11.881957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:11.881959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:11.881971 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:11.882008 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:11.882020 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:11.882092 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:11.882206 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 22:47:11.882257 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:47:11.883422 28998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e974a79fe2a44c98d6853d3f92a181e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e974a79fe2a44c98d6853d3f92a181e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35493 } } }
I20260807 22:47:11.883546 28998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:11.883894 28999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e974a79fe2a44c98d6853d3f92a181e. Latest consensus state: current_term: 1 leader_uuid: "6e974a79fe2a44c98d6853d3f92a181e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e974a79fe2a44c98d6853d3f92a181e" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 35493 } } }
I20260807 22:47:11.884045 28999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e [sys.catalog]: This master's current role is: LEADER
W20260807 22:47:11.886514 29019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:11.886579 29019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:11.886679 29016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:11.887343 29016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:11.892196 29016 catalog_manager.cc:1347] Generated new cluster ID: 7c8badc470df4c8fb916bf4726ad3747
I20260807 22:47:11.892256 29016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:11.904508 29016 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:11.905654 29016 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:11.917088 29016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e974a79fe2a44c98d6853d3f92a181e: Generated new TSK 0
I20260807 22:47:11.917847 29016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:11.921804 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 28893
2026-08-07T22:47:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:11 dist-test-slave-4tdj krb5kdc[29025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:11 dist-test-slave-4tdj krb5kdc[29025](info): set up 2 sockets
Aug 07 22:47:11 dist-test-slave-4tdj krb5kdc[29025](info): commencing operation
krb5kdc: starting...
W20260807 22:47:13.059263 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:13 dist-test-slave-4tdj krb5kdc[29025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:13.175050 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:32919
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:32919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:13.280952 29045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:13.281183 29045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:13.281221 29045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:13.285425 29045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:13.285480 29045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:13.285497 29045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:13.285512 29045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:13.285526 29045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:13.290584 29045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:32919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:32919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:13.291491 29045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:13.292424 29045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:13.300338 29054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:13.300474 29055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:13.300333 29057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:13.300400 29045 server_base.cc:1034] running on GCE node
I20260807 22:47:13.301070 29045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:13.302099 29045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:13.303256 29045 hybrid_clock.cc:648] HybridClock initialized: now 1786142833303236 us; error 31 us; skew 500 ppm
Aug 07 22:47:13 dist-test-slave-4tdj krb5kdc[29025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142833, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:13.305847 29045 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:13.306689 29045 webserver.cc:460] Webserver started at http://127.26.189.254:36363/ using document root <none> and password file <none>
I20260807 22:47:13.307188 29045 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:13.307253 29045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:13.307443 29045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:13.309365 29045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "f2e9a386f8144de5990eb3665ee52bdc"
format_stamp: "Formatted at 2026-08-07 22:47:13 on dist-test-slave-4tdj"
server_key: "f071579ed3fec13eeaf39e95cf1373ac"
server_key_iv: "c53c4c2e1b5e11f7642d54f53f945d8f"
server_key_version: "encryptionkey@0"
I20260807 22:47:13.309930 29045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "f2e9a386f8144de5990eb3665ee52bdc"
format_stamp: "Formatted at 2026-08-07 22:47:13 on dist-test-slave-4tdj"
server_key: "f071579ed3fec13eeaf39e95cf1373ac"
server_key_iv: "c53c4c2e1b5e11f7642d54f53f945d8f"
server_key_version: "encryptionkey@0"
I20260807 22:47:13.313514 29045 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:47:13.315832 29067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:13.316990 29045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:47:13.317103 29045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "f2e9a386f8144de5990eb3665ee52bdc"
format_stamp: "Formatted at 2026-08-07 22:47:13 on dist-test-slave-4tdj"
server_key: "f071579ed3fec13eeaf39e95cf1373ac"
server_key_iv: "c53c4c2e1b5e11f7642d54f53f945d8f"
server_key_version: "encryptionkey@0"
I20260807 22:47:13.317194 29045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:13.321506 29045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:13.324157 29045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:13.324316 29045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:13.330842 29045 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:32919
I20260807 22:47:13.330879 29155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:32919 every 8 connection(s)
I20260807 22:47:13.331691 29045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:13.334270 29157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:13.339587 29157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc: Bootstrap starting.
I20260807 22:47:13.340417 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29045
I20260807 22:47:13.340569 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:13.340864 27383 external_mini_cluster.cc:1428] Setting key da5b7db4f9d4eb14c0d9b4bfe5395986
I20260807 22:47:13.342548 29157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:13.343456 29157 log.cc:824] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:13.345589 29157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc: No bootstrap required, opened a new log
I20260807 22:47:13.346120 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:13.342054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57835 (local address 127.26.189.254:32919)
0807 22:47:13.342506 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:13.342526 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:13.342580 (+ 54us) server_negotiation.cc:407] Connection header received
0807 22:47:13.343423 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:13.343446 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:13.343815 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:13.344137 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:13.344891 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:13.344895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:13.344907 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:13.344917 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:13.344970 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:13.344989 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:13.345082 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:13.345228 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 22:47:13.345381 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260807 22:47:13.349035 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:13.349632 29157 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e9a386f8144de5990eb3665ee52bdc" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 32919 } }
I20260807 22:47:13.349848 29157 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:47:13.349913 29157 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e9a386f8144de5990eb3665ee52bdc, State: Initialized, Role: FOLLOWER
I20260807 22:47:13.350392 29157 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e9a386f8144de5990eb3665ee52bdc" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 32919 } }
I20260807 22:47:13.350533 29157 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:47:13.350605 29157 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:47:13.350701 29157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:13.351817 29157 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e9a386f8144de5990eb3665ee52bdc" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 32919 } }
I20260807 22:47:13.352176 29157 leader_election.cc:302] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2e9a386f8144de5990eb3665ee52bdc; no voters:
I20260807 22:47:13.352525 29157 leader_election.cc:288] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:47:13.352741 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:13.351028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55195 (local address 127.26.189.254:32919)
0807 22:47:13.351144 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:13.351148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:13.351360 (+ 212us) server_negotiation.cc:407] Connection header received
0807 22:47:13.351509 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:13.351511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:13.351540 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:13.351604 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:13.352304 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:13.352308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:13.352310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:47:13.352323 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:13.352350 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:47:13.352361 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:13.352428 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:13.352559 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:47:13.352599 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:47:13.352595 29172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:13.353048 29172 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [term 1 LEADER]: Becoming Leader. State: Replica: f2e9a386f8144de5990eb3665ee52bdc, State: Running, Role: LEADER
I20260807 22:47:13.353410 29172 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e9a386f8144de5990eb3665ee52bdc" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 32919 } }
I20260807 22:47:13.353827 29157 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:13.354552 29173 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2e9a386f8144de5990eb3665ee52bdc. Latest consensus state: current_term: 1 leader_uuid: "f2e9a386f8144de5990eb3665ee52bdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e9a386f8144de5990eb3665ee52bdc" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 32919 } } }
I20260807 22:47:13.354668 29173 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:13.355002 29175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2e9a386f8144de5990eb3665ee52bdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e9a386f8144de5990eb3665ee52bdc" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 32919 } } }
I20260807 22:47:13.355113 29175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:13.355139 29185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:13.358134 29185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:13.363466 29185 catalog_manager.cc:1347] Generated new cluster ID: 78ed088995324f3c864a0222d76068a7
I20260807 22:47:13.363529 29185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:13.376905 29185 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:13.377218 29185 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:13.384828 29185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2e9a386f8144de5990eb3665ee52bdc: Generated new TSK 0
I20260807 22:47:13.385557 29185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:47:13.396710 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29045
2026-08-07T22:47:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1472 ms)
[----------] 18 tests from AuthTokenIssuingTest (26873 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:13 dist-test-slave-4tdj krb5kdc[29201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:13 dist-test-slave-4tdj krb5kdc[29201](info): set up 2 sockets
Aug 07 22:47:13 dist-test-slave-4tdj krb5kdc[29201](info): commencing operation
krb5kdc: starting...
W20260807 22:47:14.538208 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:14 dist-test-slave-4tdj krb5kdc[29201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:14.656946 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:41655
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:41655,127.26.189.253:35563,127.26.189.252:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:14.764555 29219 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:47:14.764823 29219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:14.764858 29219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:14.764919 29219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:14.768858 29219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:14.768910 29219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:14.768936 29219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:14.768957 29219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:14.768983 29219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:14.774082 29219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:41655,127.26.189.253:35563,127.26.189.252:45503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:41655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:14.775086 29219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:14.776034 29219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:14.784287 29226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:14.784435 29219 server_base.cc:1034] running on GCE node
W20260807 22:47:14.784282 29228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:14.784281 29225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:14.785030 29219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:14.786095 29219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:14.787248 29219 hybrid_clock.cc:648] HybridClock initialized: now 1786142834787220 us; error 40 us; skew 500 ppm
Aug 07 22:47:14 dist-test-slave-4tdj krb5kdc[29201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:14.789786 29219 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:14.790699 29219 webserver.cc:460] Webserver started at http://127.26.189.254:32795/ using document root <none> and password file <none>
I20260807 22:47:14.791211 29219 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:14.791277 29219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:14.791471 29219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:14.793453 29219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "2efda708925e4b439d7e9e1045758824"
format_stamp: "Formatted at 2026-08-07 22:47:14 on dist-test-slave-4tdj"
server_key: "4b12cdb7489015b9a844f2d331f841aa"
server_key_iv: "ffba854f92bafdaa180de0279f73c89e"
server_key_version: "encryptionkey@0"
I20260807 22:47:14.793982 29219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "2efda708925e4b439d7e9e1045758824"
format_stamp: "Formatted at 2026-08-07 22:47:14 on dist-test-slave-4tdj"
server_key: "4b12cdb7489015b9a844f2d331f841aa"
server_key_iv: "ffba854f92bafdaa180de0279f73c89e"
server_key_version: "encryptionkey@0"
I20260807 22:47:14.797429 29219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:47:14.799680 29237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:14.800803 29219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:47:14.800925 29219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "2efda708925e4b439d7e9e1045758824"
format_stamp: "Formatted at 2026-08-07 22:47:14 on dist-test-slave-4tdj"
server_key: "4b12cdb7489015b9a844f2d331f841aa"
server_key_iv: "ffba854f92bafdaa180de0279f73c89e"
server_key_version: "encryptionkey@0"
I20260807 22:47:14.801015 29219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:14.816430 29219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:14.819093 29219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:14.819254 29219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:14.825925 29219 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:41655
I20260807 22:47:14.826192 29316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:41655 every 8 connection(s)
I20260807 22:47:14.826841 29219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:14.829438 29318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:14.832623 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29219
I20260807 22:47:14.832732 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:14.832973 27383 external_mini_cluster.cc:1428] Setting key 6138e79d62ba3f93826ed8f91bd26b80
I20260807 22:47:14.833107 29318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.189.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.253@KRBTEST.COM" created.
Aug 07 22:47:14 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:14.850328 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.835292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.26.189.254:41655)
0807 22:47:14.835838 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:14.835856 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:14.835990 (+ 134us) server_negotiation.cc:407] Connection header received
0807 22:47:14.837152 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:14.837157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:14.837330 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:14.837473 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:14.838501 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.839371 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.840784 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.841139 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.841248 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:14.845806 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:14.845811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:14.845821 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:14.845842 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:14.847563 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.848015 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.848021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.848023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.848166 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:14.848424 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:14.848427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:14.848428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:14.848933 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:14.849082 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:14.849395 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 22:47:14.849638 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":427,"spinlock_wait_cycles":54144,"thread_start_us":146,"threads_started":1}
I20260807 22:47:14.850337 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.835292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:41655 (local address 127.0.0.1:37930)
0807 22:47:14.835832 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.835885 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:14.836870 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:14.837542 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:14.837547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:14.837980 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:14.838362 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:14.838383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.839497 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.839501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:14.840677 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:14.840690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:14.841296 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:14.841302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:14.841417 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:14.841993 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:14.842015 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:14.845664 (+ 3649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:14.847673 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:14.847682 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:14.847689 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:14.847904 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:14.848248 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:14.848250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:14.848252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:14.848348 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:14.849039 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:14.849044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:14.849165 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 22:47:14.849298 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 22:47:14.849533 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":138,"threads_started":1}
I20260807 22:47:14.852789 29318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:14.853415 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.853193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37316)
0807 22:47:14.853296 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.853364 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 22:47:14.853587 29239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
I20260807 22:47:14.854355 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.854146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37330)
0807 22:47:14.854233 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.854277 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 22:47:14.854489 29318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:35563: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab.
Entry for principal kudu/127.26.189.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab.
I20260807 22:47:14.859287 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:35563
--webserver_interface=127.26.189.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab
--principal=kudu/127.26.189.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:41655,127.26.189.253:35563,127.26.189.252:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:47:14.903769 29318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } attempt: 1
I20260807 22:47:14.904681 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.904358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37334)
0807 22:47:14.904527 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.904598 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 22:47:14.905462 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.905195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37350)
0807 22:47:14.905321 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.905379 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 22:47:14.905650 29318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:35563: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
W20260807 22:47:14.967630 29329 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:47:14.967900 29329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:14.967936 29329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:14.968005 29329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:14.972005 29329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:14.972059 29329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:14.972084 29329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:14.972105 29329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:14.972134 29329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:14.973821 29318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } attempt: 2
I20260807 22:47:14.974612 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.974327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37358)
0807 22:47:14.974463 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.974528 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 22:47:14.975332 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:14.975121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37372)
0807 22:47:14.975230 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:14.975273 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:47:14.975440 29318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:35563: Network error: Client connection negotiation failed: client connection to 127.26.189.253:35563: connect: Connection refused (error 111)
I20260807 22:47:14.977445 29329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:41655,127.26.189.253:35563,127.26.189.252:45503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab
--principal=kudu/127.26.189.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:35563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.189.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:14.978471 29329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:14.979430 29329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:14.987442 29341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:14.987429 29339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:14.987432 29337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:14.987484 29329 server_base.cc:1034] running on GCE node
I20260807 22:47:14.988157 29329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:14.989205 29329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:14.990367 29329 hybrid_clock.cc:648] HybridClock initialized: now 1786142834990341 us; error 39 us; skew 500 ppm
Aug 07 22:47:14 dist-test-slave-4tdj krb5kdc[29201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:14.993012 29329 init.cc:377] Logged in from keytab as kudu/127.26.189.253@KRBTEST.COM (short username kudu)
I20260807 22:47:14.993841 29329 webserver.cc:460] Webserver started at http://127.26.189.253:34081/ using document root <none> and password file <none>
I20260807 22:47:14.994387 29329 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:14.994454 29329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:14.994622 29329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:14.996543 29329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data/instance:
uuid: "fe7198380fbb429996c7867556d8fe8f"
format_stamp: "Formatted at 2026-08-07 22:47:14 on dist-test-slave-4tdj"
server_key: "5af081b65557cf27c0bfb8990036af85"
server_key_iv: "51b264b00bb5ff4a11fc62c2480144b5"
server_key_version: "encryptionkey@0"
I20260807 22:47:14.997038 29329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal/instance:
uuid: "fe7198380fbb429996c7867556d8fe8f"
format_stamp: "Formatted at 2026-08-07 22:47:14 on dist-test-slave-4tdj"
server_key: "5af081b65557cf27c0bfb8990036af85"
server_key_iv: "51b264b00bb5ff4a11fc62c2480144b5"
server_key_version: "encryptionkey@0"
I20260807 22:47:15.000469 29329 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 22:47:15.002781 29350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:15.003937 29329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 22:47:15.004050 29329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal
uuid: "fe7198380fbb429996c7867556d8fe8f"
format_stamp: "Formatted at 2026-08-07 22:47:14 on dist-test-slave-4tdj"
server_key: "5af081b65557cf27c0bfb8990036af85"
server_key_iv: "51b264b00bb5ff4a11fc62c2480144b5"
server_key_version: "encryptionkey@0"
I20260807 22:47:15.004148 29329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:15.022372 29329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:15.026441 29329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:15.026705 29329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:15.039584 29329 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.253:35563
I20260807 22:47:15.040611 29329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
I20260807 22:47:15.041020 29422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.253:35563 every 8 connection(s)
I20260807 22:47:15.044258 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:15.045252 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29329
I20260807 22:47:15.045405 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-1/wal/instance
I20260807 22:47:15.045778 27383 external_mini_cluster.cc:1428] Setting key 70daab9c7f7de50dea9592b32a1c85af
I20260807 22:47:15.048820 29424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.189.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.252@KRBTEST.COM" created.
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:15.064455 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.052280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.26.189.254:41655)
0807 22:47:15.052438 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.052444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.052479 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:47:15.053024 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.053027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.053088 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.053165 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.054360 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.054876 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.055970 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.056214 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.056309 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.059909 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.059912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.059914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.059937 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.062338 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.062852 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.062856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.062858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.062969 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.063285 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.063288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.063290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.063533 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.063636 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.064140 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.064278 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:47:15.065116 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.051284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:41655 (local address 127.0.0.1:37936)
0807 22:47:15.051764 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.051816 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.052776 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.053350 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.053360 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.053744 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.054202 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.054215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.054995 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.054999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.055854 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.055864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.056384 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.056390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.056598 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.057141 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.057159 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.059743 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.062452 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.062458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.062470 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.062740 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.063060 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.063062 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.063064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.063190 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.063631 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.063636 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.063873 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.064037 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.064281 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
I20260807 22:47:15.066565 29424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:15.066583 29318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } attempt: 3
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab.
Entry for principal kudu/127.26.189.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab.
I20260807 22:47:15.075700 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:45503
--webserver_interface=127.26.189.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab
--principal=kudu/127.26.189.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:41655,127.26.189.253:35563,127.26.189.252:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:47:15.076609 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.066958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37384)
0807 22:47:15.067109 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.067124 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.067202 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.068028 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.068031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.068195 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.068307 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.068314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.069673 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.069676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.070250 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.070260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.070973 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.070977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.071085 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.071605 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.071616 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.073112 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.074943 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.074946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.074948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.075185 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.075525 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.075530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.075634 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.076146 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.076149 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.076242 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.076343 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.076458 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 22:47:15.076602 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.067055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37396)
0807 22:47:15.067260 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.067272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.067346 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.068028 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.068031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.068239 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.068337 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.068343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.069702 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.069706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.070383 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.070394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.070923 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.070926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.071059 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.071455 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.071469 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.072718 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.075255 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.075260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.075432 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.075719 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.075723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.075801 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.076170 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.076173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.076241 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.076320 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.076440 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":31}
I20260807 22:47:15.076773 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.067185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.26.189.253:35563)
0807 22:47:15.067581 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.067592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.067614 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:47:15.067655 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.067660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.067886 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.068023 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.068419 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.069555 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.070373 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.070865 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.070977 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.073252 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.073255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.073257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.073301 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.074846 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075316 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.075321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.075323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.075412 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075732 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.075736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.075738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.076024 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.076162 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.076432 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.076573 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":299,"server-negotiator.queue_time_us":299,"spinlock_wait_cycles":10496,"thread_start_us":128,"threads_started":1}
I20260807 22:47:15.077231 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.067396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.26.189.253:35563)
0807 22:47:15.067641 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.067649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.067659 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:47:15.067682 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.067684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.067885 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.068023 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.068441 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.069599 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.070482 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.070822 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.070914 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.073006 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.073016 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.073027 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.073048 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.075158 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075521 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.075524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.075526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.075626 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.075900 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.075903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.075905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.076082 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.076887 (+ 805us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.076957 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.077075 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":194,"spinlock_wait_cycles":12160,"thread_start_us":107,"threads_started":1}
I20260807 22:47:15.078835 29318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:15.078903 29424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:15.079535 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.079296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57054)
0807 22:47:15.079402 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.079475 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 22:47:15.079617 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.079319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57058)
0807 22:47:15.079505 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.079554 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260807 22:47:15.079757 29355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
I20260807 22:47:15.080334 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.080016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57068)
0807 22:47:15.080223 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.080275 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260807 22:47:15.080525 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.080341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57070)
0807 22:47:15.080431 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.080476 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:47:15.080539 29318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:45503: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
W20260807 22:47:15.080708 29424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:45503: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
I20260807 22:47:15.111694 29318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } attempt: 1
I20260807 22:47:15.112557 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.112225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57084)
0807 22:47:15.112385 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.112459 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 22:47:15.113257 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.113033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57096)
0807 22:47:15.113137 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.113188 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:47:15.113389 29318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:45503: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
I20260807 22:47:15.130004 29424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } attempt: 1
I20260807 22:47:15.130879 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.130592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57108)
0807 22:47:15.130733 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.130794 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 22:47:15.131552 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.131346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57118)
0807 22:47:15.131446 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.131493 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 22:47:15.131702 29424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:45503: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
W20260807 22:47:15.187870 29437 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:47:15.188123 29437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:15.188165 29437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:15.188225 29437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 22:47:15.188526 29318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } attempt: 2
I20260807 22:47:15.189323 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.189008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57120)
0807 22:47:15.189166 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.189235 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 22:47:15.189997 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.189796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57134)
0807 22:47:15.189886 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.189935 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:47:15.190137 29318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:45503: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
W20260807 22:47:15.192643 29437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:15.192693 29437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:15.192710 29437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:15.192729 29437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:15.192759 29437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:15.197873 29437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:41655,127.26.189.253:35563,127.26.189.252:45503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab
--principal=kudu/127.26.189.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:45503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.189.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:15.198874 29437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:15.199836 29437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:47:15.199832 29424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } attempt: 2
I20260807 22:47:15.200510 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.200218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57136)
0807 22:47:15.200367 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.200431 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 22:47:15.201187 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.200956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57148)
0807 22:47:15.201062 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.201123 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 22:47:15.201314 29424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:45503: Network error: Client connection negotiation failed: client connection to 127.26.189.252:45503: connect: Connection refused (error 111)
W20260807 22:47:15.208046 29449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:15.208033 29448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:15.208129 29437 server_base.cc:1034] running on GCE node
W20260807 22:47:15.208012 29452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:15.208834 29437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:15.209856 29437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:15.211006 29437 hybrid_clock.cc:648] HybridClock initialized: now 1786142835210986 us; error 43 us; skew 500 ppm
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142835, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:15.213708 29437 init.cc:377] Logged in from keytab as kudu/127.26.189.252@KRBTEST.COM (short username kudu)
I20260807 22:47:15.214540 29437 webserver.cc:460] Webserver started at http://127.26.189.252:45435/ using document root <none> and password file <none>
I20260807 22:47:15.215021 29437 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:15.215080 29437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:15.215241 29437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:15.217137 29437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data/instance:
uuid: "df7a318acc504ce5a48f35ea709d8f23"
format_stamp: "Formatted at 2026-08-07 22:47:15 on dist-test-slave-4tdj"
server_key: "ccbc552cb2119495271e5c20951e1f8b"
server_key_iv: "988fc020c031c399e19d94118a862acf"
server_key_version: "encryptionkey@0"
I20260807 22:47:15.217638 29437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal/instance:
uuid: "df7a318acc504ce5a48f35ea709d8f23"
format_stamp: "Formatted at 2026-08-07 22:47:15 on dist-test-slave-4tdj"
server_key: "ccbc552cb2119495271e5c20951e1f8b"
server_key_iv: "988fc020c031c399e19d94118a862acf"
server_key_version: "encryptionkey@0"
I20260807 22:47:15.221109 29437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:47:15.223320 29463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:15.224444 29437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:47:15.224547 29437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal
uuid: "df7a318acc504ce5a48f35ea709d8f23"
format_stamp: "Formatted at 2026-08-07 22:47:15 on dist-test-slave-4tdj"
server_key: "ccbc552cb2119495271e5c20951e1f8b"
server_key_iv: "988fc020c031c399e19d94118a862acf"
server_key_version: "encryptionkey@0"
I20260807 22:47:15.224627 29437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:15.241494 29437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:15.244310 29437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:15.244454 29437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:15.250701 29437 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.252:45503
I20260807 22:47:15.250941 29543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.252:45503 every 8 connection(s)
I20260807 22:47:15.251575 29437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
I20260807 22:47:15.252974 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29437
I20260807 22:47:15.253103 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142556986508-27383-0/minicluster-data/master-2/wal/instance
I20260807 22:47:15.253414 27383 external_mini_cluster.cc:1428] Setting key e6967f06983bbebf0d34760abf3435a1
I20260807 22:47:15.254666 29544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:15.259140 29544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142835, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:15.267675 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.254743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.26.189.252:45503)
0807 22:47:15.255139 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.255156 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.255198 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:47:15.256038 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.256061 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.256443 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.256791 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.257440 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.258893 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.259794 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.260427 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.260565 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.263175 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.263180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.263188 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.263213 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.264945 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.265396 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.265400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.265404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.265508 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.265794 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.265798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.265799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.266149 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.266318 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.266652 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.266912 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":117,"threads_started":1}
I20260807 22:47:15.269222 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.260875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.26.189.254:41655)
0807 22:47:15.261033 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.261038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.261398 (+ 360us) server_negotiation.cc:407] Connection header received
0807 22:47:15.261517 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.261520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.261575 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.261667 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.262262 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.262840 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.263757 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.263992 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.264088 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.266133 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.266136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.266139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.266165 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.267539 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.268040 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.268044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.268046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.268136 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.268379 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.268382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.268384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.268568 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.268656 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.268959 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.269061 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:47:15.269470 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.260852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:41655 (local address 127.0.0.1:37952)
0807 22:47:15.261266 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.261311 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.261424 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.261702 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.261706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.262010 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.262140 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.262148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.262955 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.262958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.263640 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.263652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.264091 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.264095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.264204 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.264642 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.264653 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.265997 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.267642 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.267652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.267657 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.267950 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.268218 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.268221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.268223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.268300 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.268670 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.268676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.268779 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.268893 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.269058 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":162,"threads_started":1}
I20260807 22:47:15.270422 29544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142835, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:47:15.277746 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.269680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.26.189.254:41655)
0807 22:47:15.269880 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.269883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.269953 (+ 70us) server_negotiation.cc:407] Connection header received
0807 22:47:15.270073 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.270076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.270116 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.270171 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.270875 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.271335 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.272125 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.272376 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.272446 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.274770 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.274774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.274777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.274802 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.276286 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.276702 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.276705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.276706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.276775 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.277012 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.277015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.277017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.277223 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.277433 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.277517 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.277619 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260807 22:47:15.277920 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.270852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:35563 (local address 127.0.0.1:37398)
0807 22:47:15.270996 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.271007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.271070 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.271285 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.271288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.271436 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.271533 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.271540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.272288 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.272290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.272976 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.272988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.273420 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.273423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.273525 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.273889 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.273900 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.275014 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.276578 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.276582 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.276584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.276785 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.277040 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.277043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.277044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.277114 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.277492 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.277495 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.277581 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.277680 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.277781 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 22:47:15.278124 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.270960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37398 (local address 127.26.189.253:35563)
0807 22:47:15.271095 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.271100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.271114 (+ 14us) server_negotiation.cc:407] Connection header received
0807 22:47:15.271153 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.271156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.271197 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.271259 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.271662 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.272187 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.273092 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.273317 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.273393 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.275150 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.275153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.275156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.275180 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.276479 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.276872 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.276876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.276878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.276950 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.277195 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.277199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.277201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.277398 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.277490 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.277864 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.277982 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:47:15.278782 29544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142835, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:47:15.286725 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.279325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57168)
0807 22:47:15.279420 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.279433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.279498 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.279752 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.279754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.279902 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.279982 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.279988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.280804 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.280807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.281380 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.281389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.281927 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.281931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.282000 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.282397 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.282407 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.283512 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.285139 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.285142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.285144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.285381 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.285683 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.285686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.285688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.285775 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.286146 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.286149 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.286240 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.286390 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.286516 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:47:15.286861 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.279450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.26.189.252:45503)
0807 22:47:15.279566 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.279578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.279590 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:47:15.279621 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.279624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.279658 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.279745 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.280085 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.280695 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.281488 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.281829 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.281917 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.283639 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.283642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.283644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.283664 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.285034 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.285510 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.285512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.285514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.285597 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.285860 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.285862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.285863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.286043 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.286142 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.286434 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.286705 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":39}
I20260807 22:47:15.287218 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.278825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.26.189.253:35563)
0807 22:47:15.278926 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.278930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.278942 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:47:15.279078 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.279082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.279122 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.279191 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.279800 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.280355 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.281172 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.281474 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.281561 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.284179 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.284182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.284185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.284207 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.285565 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.285980 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.285983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.285985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.286075 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.286356 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.286359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.286360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.286561 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.286756 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.286892 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.287102 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 22:47:15.289317 29318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } attempt: 3
I20260807 22:47:15.290364 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:15.290516 29544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: Bootstrap starting.
I20260807 22:47:15.292632 29424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } attempt: 3
I20260807 22:47:15.291682 29281 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2efda708925e4b439d7e9e1045758824"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37962
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:47:15.293886 29544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:15.294864 29544 log.cc:824] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:15.297089 29544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: No bootstrap required, opened a new log
I20260807 22:47:15.297302 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.289706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57174)
0807 22:47:15.289858 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.289870 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.289933 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.290311 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.290315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.290477 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.290568 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.290574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.291232 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.291235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.291731 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.291739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.292134 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.292137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.292215 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.292656 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.292670 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.294017 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.295775 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.295778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.295780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.296015 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.296302 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.296305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.296307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.296402 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.296840 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.296843 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.296938 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.297036 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.297162 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 22:47:15.297827 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.290030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.26.189.252:45503)
0807 22:47:15.290130 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.290133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.290143 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:47:15.290167 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.290169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.290196 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.290258 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.290690 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.291135 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.291833 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.292043 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.292122 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.294166 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.294170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.294172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.294192 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.295686 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.296128 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.296131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.296133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.296215 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.296490 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.296493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.296495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.296702 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.296790 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.297602 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.297705 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 22:47:15 dist-test-slave-4tdj krb5kdc[29201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142834, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:47:15.300885 29318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824: Bootstrap starting.
I20260807 22:47:15.301107 29544 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.301355 29544 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7a318acc504ce5a48f35ea709d8f23, State: Initialized, Role: FOLLOWER
I20260807 22:47:15.301688 29544 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.302367 29557 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.302505 29557 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.302672 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.293208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:45503 (local address 127.0.0.1:57180)
0807 22:47:15.293419 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:15.293432 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:15.293501 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:15.294174 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:15.294177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:15.294369 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:15.294458 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.294465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.295470 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.295473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.296209 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:15.296218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.296988 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.296991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:15.297078 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.297546 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:15.297561 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:15.299110 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:15.301052 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.301056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.301071 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.301312 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.301624 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:15.301627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:15.301629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:15.301726 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:15.302139 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:15.302142 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:15.302245 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 22:47:15.302392 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:47:15.302516 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260807 22:47:15.303062 29544 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:15.303243 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.293641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.26.189.252:45503)
0807 22:47:15.293959 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.293968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.293981 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:47:15.294013 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.294015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.294054 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.294131 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.294582 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.295356 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.296512 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.296887 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.296973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.299258 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.299262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.299264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.299288 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.300894 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.301420 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.301423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.301425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.301526 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.301818 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.301821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.301823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.302029 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.302137 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.302966 (+ 829us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.303080 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":97,"threads_started":1}
I20260807 22:47:15.303587 29318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:15.304368 29318 log.cc:824] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:15.306136 29424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: Bootstrap starting.
I20260807 22:47:15.306493 29318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824: No bootstrap required, opened a new log
I20260807 22:47:15.309056 29424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:15.309378 29318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.309602 29318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2efda708925e4b439d7e9e1045758824, State: Initialized, Role: FOLLOWER
I20260807 22:47:15.309870 29424 log.cc:824] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:15.309911 29318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.310523 29581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.310664 29581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.311175 29318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:15.311262 29281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:47:15.311359 29281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:15.308708 29437 master_runner.cc:186] Error getting master registration for 127.26.189.254:41655: OK, instance_id { permanent_uuid: "2efda708925e4b439d7e9e1045758824" instance_seqno: 1786142834824053 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:47:15.312206 29437 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:47:15.313009 29281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.313817 29424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: No bootstrap required, opened a new log
I20260807 22:47:15.313951 29281 leader_election.cc:288] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [CANDIDATE]: Term 1 election: Requested vote from peers fe7198380fbb429996c7867556d8fe8f (127.26.189.253:35563), df7a318acc504ce5a48f35ea709d8f23 (127.26.189.252:45503)
W20260807 22:47:15.314854 29580 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:15.314922 29580 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:15.316430 29505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2efda708925e4b439d7e9e1045758824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "df7a318acc504ce5a48f35ea709d8f23"
I20260807 22:47:15.316689 29505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:15.317710 29424 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.317921 29424 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe7198380fbb429996c7867556d8fe8f, State: Initialized, Role: FOLLOWER
I20260807 22:47:15.318002 29505 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2efda708925e4b439d7e9e1045758824 in term 1.
I20260807 22:47:15.318243 29424 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.318542 29239 leader_election.cc:302] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2efda708925e4b439d7e9e1045758824, df7a318acc504ce5a48f35ea709d8f23; no voters:
I20260807 22:47:15.318811 29581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:15.318758 29606 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.318893 29606 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.319015 29581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [term 1 LEADER]: Becoming Leader. State: Replica: 2efda708925e4b439d7e9e1045758824, State: Running, Role: LEADER
I20260807 22:47:15.319306 29581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } }
I20260807 22:47:15.319578 29424 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: configured and running, proceeding with master startup.
W20260807 22:47:15.320937 29247 tablet.cc:2369] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:47:15.321026 29386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2efda708925e4b439d7e9e1045758824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fe7198380fbb429996c7867556d8fe8f"
I20260807 22:47:15.321228 29386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:15.321738 29608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efda708925e4b439d7e9e1045758824. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.321843 29608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:15.322564 29386 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2efda708925e4b439d7e9e1045758824 in term 1.
I20260807 22:47:15.322620 29635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:15.329212 29635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 22:47:15.332026 29638 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:15.332116 29638 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:15.333299 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.318370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.26.189.254:41655)
0807 22:47:15.318499 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.318504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.320079 (+ 1575us) server_negotiation.cc:407] Connection header received
0807 22:47:15.320273 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.320276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.320325 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.320377 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.321389 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.322121 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.323262 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.323566 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.323656 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.327023 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.327027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.327030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.327054 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.330342 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.331886 (+ 1544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.331890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.331892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.331980 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.332301 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.332304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.332306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.332533 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.332657 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.333021 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.333148 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:47:15.336746 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.317186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57188 (local address 127.26.189.252:45503)
0807 22:47:15.317287 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.317291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.328211 (+ 10920us) server_negotiation.cc:407] Connection header received
0807 22:47:15.328552 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.328555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.328604 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.328654 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.329458 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.330186 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.331255 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.331615 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.331687 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.333283 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.333288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.333290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.333314 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.335079 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.335539 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.335543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.335545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.335647 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.335898 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.335902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.335904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.336120 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.336245 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.336471 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.336600 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:47:15.338800 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:15.316409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.26.189.253:35563)
0807 22:47:15.316527 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.316531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.328298 (+ 11767us) server_negotiation.cc:407] Connection header received
0807 22:47:15.328413 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.328416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.328460 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.328532 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.330008 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.330717 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.333841 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:15.334178 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:15.334248 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:15.335729 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:15.335732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:15.335735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:15.335758 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:15.337222 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.337683 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.337686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.337688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.337762 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:15.338017 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:15.338018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:15.338020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:15.338164 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:15.338261 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:15.338566 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 22:47:15.338671 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:47:15.345904 29505 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [term 1 FOLLOWER]: Refusing update from remote peer 2efda708925e4b439d7e9e1045758824: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:15.347133 29608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:15.352949 29557 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efda708925e4b439d7e9e1045758824. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.353072 29557 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.354439 29386 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [term 1 FOLLOWER]: Refusing update from remote peer 2efda708925e4b439d7e9e1045758824: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:15.355028 29608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:15.358348 29606 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efda708925e4b439d7e9e1045758824. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.358453 29606 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.364501 29581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.364614 29581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:15.364697 29581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.364754 29581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824 [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:15.366052 29557 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.366155 29557 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.367116 29606 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2efda708925e4b439d7e9e1045758824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efda708925e4b439d7e9e1045758824" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 41655 } } peers { permanent_uuid: "fe7198380fbb429996c7867556d8fe8f" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 35563 } } peers { permanent_uuid: "df7a318acc504ce5a48f35ea709d8f23" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 45503 } } }
I20260807 22:47:15.367234 29606 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:15.367825 29641 mvcc.cc:205] Tried to move back new op lower bound from 7316041053568806912 to 7316041053471027200. Current Snapshot: MvccSnapshot[applied={T|T < 7316041053568806912}]
I20260807 22:47:15.368037 29642 mvcc.cc:205] Tried to move back new op lower bound from 7316041053568806912 to 7316041053471027200. Current Snapshot: MvccSnapshot[applied={T|T < 7316041053568806912}]
I20260807 22:47:15.370457 29645 mvcc.cc:205] Tried to move back new op lower bound from 7316041053568806912 to 7316041053471027200. Current Snapshot: MvccSnapshot[applied={T|T < 7316041053568806912}]
I20260807 22:47:15.372136 29635 catalog_manager.cc:1347] Generated new cluster ID: 54f0ee0c5e1f475e9ec530e1f65528d4
I20260807 22:47:15.372191 29635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:15.397598 29635 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:15.400905 29635 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:15.417284 29635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2efda708925e4b439d7e9e1045758824: Generated new TSK 0
I20260807 22:47:15.418215 29635 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:47:15.454593 29324 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.451407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.26.189.254:41655)
0807 22:47:15.451537 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.451542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.453438 (+ 1896us) server_negotiation.cc:407] Connection header received
0807 22:47:15.453652 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.453656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.453711 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.453767 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.454462 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37990: BlockingRecv error: recv got EOF from 127.0.0.1:37990 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 22:47:15.456234 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.452283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37428 (local address 127.26.189.253:35563)
0807 22:47:15.452403 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.452407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.455274 (+ 2867us) server_negotiation.cc:407] Connection header received
0807 22:47:15.455456 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.455459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.455510 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.455562 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.456104 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37428: BlockingRecv error: recv got EOF from 127.0.0.1:37428 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 22:47:15.457955 29552 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.452920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.26.189.252:45503)
0807 22:47:15.453061 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.453066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.456934 (+ 3868us) server_negotiation.cc:407] Connection header received
0807 22:47:15.457241 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.457244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.457299 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.457351 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.457838 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57190: BlockingRecv error: recv got EOF from 127.0.0.1:57190 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 22:47:15.462039 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.460580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37436 (local address 127.26.189.253:35563)
0807 22:47:15.460682 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.460687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.461127 (+ 440us) server_negotiation.cc:407] Connection header received
0807 22:47:15.461336 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.461339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.461411 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.461461 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.461931 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37436: BlockingRecv error: recv got EOF from 127.0.0.1:37436 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 22:47:15.468044 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.466675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.26.189.253:35563)
0807 22:47:15.466765 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.466769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.467220 (+ 451us) server_negotiation.cc:407] Connection header received
0807 22:47:15.467393 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.467396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.467442 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.467492 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.467944 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37440: BlockingRecv error: recv got EOF from 127.0.0.1:37440 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 22:47:15.473605 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.472138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.26.189.253:35563)
0807 22:47:15.472233 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.472236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.472737 (+ 501us) server_negotiation.cc:407] Connection header received
0807 22:47:15.472931 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.472934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.472994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.473044 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.473517 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37446: BlockingRecv error: recv got EOF from 127.0.0.1:37446 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 22:47:15.481357 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.479810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.26.189.253:35563)
0807 22:47:15.479939 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.479942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.480470 (+ 528us) server_negotiation.cc:407] Connection header received
0807 22:47:15.480663 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.480666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.480712 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.480760 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.481268 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37450: BlockingRecv error: recv got EOF from 127.0.0.1:37450 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 22:47:15.493357 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.491732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37460 (local address 127.26.189.253:35563)
0807 22:47:15.491841 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.491845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.492406 (+ 561us) server_negotiation.cc:407] Connection header received
0807 22:47:15.492621 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.492625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.492677 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.492732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.493258 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37460: BlockingRecv error: recv got EOF from 127.0.0.1:37460 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 22:47:15.513502 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.511758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.26.189.253:35563)
0807 22:47:15.511863 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.511868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.512472 (+ 604us) server_negotiation.cc:407] Connection header received
0807 22:47:15.512744 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.512748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.512802 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.512860 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.513399 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37472: BlockingRecv error: recv got EOF from 127.0.0.1:37472 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 22:47:15.548956 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.547908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.26.189.253:35563)
0807 22:47:15.548050 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.548055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.548199 (+ 144us) server_negotiation.cc:407] Connection header received
0807 22:47:15.548322 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.548324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.548370 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.548438 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.548842 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37484: BlockingRecv error: recv got EOF from 127.0.0.1:37484 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 22:47:15.616076 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.615075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.26.189.253:35563)
0807 22:47:15.615192 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.615196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.615342 (+ 146us) server_negotiation.cc:407] Connection header received
0807 22:47:15.615490 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.615493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.615544 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.615611 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.615964 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37500: BlockingRecv error: recv got EOF from 127.0.0.1:37500 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 22:47:15.747210 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:15.746123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.26.189.253:35563)
0807 22:47:15.746249 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:15.746253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:15.746372 (+ 119us) server_negotiation.cc:407] Connection header received
0807 22:47:15.746545 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:15.746549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:15.746622 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:15.746713 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:15.747109 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37506: BlockingRecv error: recv got EOF from 127.0.0.1:37506 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 22:47:16.006599 29435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:16.005494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37522 (local address 127.26.189.253:35563)
0807 22:47:16.005629 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.005634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.005698 (+ 64us) server_negotiation.cc:407] Connection header received
0807 22:47:16.005829 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.005832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.005884 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.005963 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:16.006477 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37522: BlockingRecv error: recv got EOF from 127.0.0.1:37522 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:47:16.316206 29580 catalog_manager.cc:1259] Loaded cluster ID: 54f0ee0c5e1f475e9ec530e1f65528d4
I20260807 22:47:16.316293 29580 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: loading cluster ID for follower catalog manager: success
I20260807 22:47:16.317950 29580 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: acquiring CA information for follower catalog manager: success
I20260807 22:47:16.319166 29580 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df7a318acc504ce5a48f35ea709d8f23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:47:16.333267 29638 catalog_manager.cc:1259] Loaded cluster ID: 54f0ee0c5e1f475e9ec530e1f65528d4
I20260807 22:47:16.333333 29638 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: loading cluster ID for follower catalog manager: success
I20260807 22:47:16.334882 29638 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: acquiring CA information for follower catalog manager: success
I20260807 22:47:16.335934 29638 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe7198380fbb429996c7867556d8fe8f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:47:16.524780 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:16.520646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.26.189.253:35563)
0807 22:47:16.520904 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.520908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.521091 (+ 183us) server_negotiation.cc:407] Connection header received
0807 22:47:16.521243 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.521246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.521372 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.521471 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:16.521807 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.522381 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.523267 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.523502 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.523595 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:16.523924 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:16.524381 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:16.524494 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:16.524568 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 22:47:16.524626 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260807 22:47:16.531078 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:16.527812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.26.189.252:45503)
0807 22:47:16.528084 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.528088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.528100 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:47:16.528176 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.528178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.528301 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.528385 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:16.528561 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.529058 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.529821 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.530043 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.530116 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:16.530369 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:16.530749 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:16.530834 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:16.530885 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 22:47:16.530930 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
I20260807 22:47:16.538144 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:16.533819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.26.189.253:35563)
0807 22:47:16.533950 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.533954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.534269 (+ 315us) server_negotiation.cc:407] Connection header received
0807 22:47:16.534517 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.534520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.534565 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.534639 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:16.535074 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.535810 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.537015 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.537346 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.537435 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:16.537693 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:16.537767 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:16.537827 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:16.537965 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:47:16.538013 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:47:16.538586 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:16.534193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.26.189.252:45503)
0807 22:47:16.534334 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.534337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.534549 (+ 212us) server_negotiation.cc:407] Connection header received
0807 22:47:16.534633 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.534636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.534677 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.534735 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:16.534968 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.535734 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.537493 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.537727 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.537825 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:16.538103 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:16.538182 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:16.538228 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:16.538424 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 22:47:16.538475 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:47:16.539422 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:16.533818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.26.189.254:41655)
0807 22:47:16.534084 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:16.534088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:16.534314 (+ 226us) server_negotiation.cc:407] Connection header received
0807 22:47:16.534419 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:16.534422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:16.534568 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:16.534667 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:16.535932 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.536450 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.537407 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:16.537772 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:16.537868 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:16.538365 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:16.538702 (+ 337us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:16.538797 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:16.539221 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 22:47:16.539287 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":78,"threads_started":1}
I20260807 22:47:16.542181 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29219
I20260807 22:47:16.554106 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29329
I20260807 22:47:16.563642 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29437
2026-08-07T22:47:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3164 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:47:16 dist-test-slave-4tdj krb5kdc[29779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:47:16 dist-test-slave-4tdj krb5kdc[29779](info): set up 2 sockets
Aug 07 22:47:16 dist-test-slave-4tdj krb5kdc[29779](info): commencing operation
krb5kdc: starting...
W20260807 22:47:17.701943 27383 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/test-user.keytab.
Aug 07 22:47:17 dist-test-slave-4tdj krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:47:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:47:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.189.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
Entry for principal kudu/127.26.189.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab.
I20260807 22:47:17.824441 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36835
--webserver_interface=127.26.189.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36835,127.26.189.253:39483,127.26.189.252:37805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:47:17.935535 29800 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:47:17.935788 29800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:17.935809 29800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:17.935856 29800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:17.939788 29800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:17.939836 29800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:17.939854 29800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:17.939869 29800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:17.939884 29800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:17.944963 29800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36835,127.26.189.253:39483,127.26.189.252:37805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.254.keytab
--principal=kudu/127.26.189.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.254:36835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.189.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:17.945966 29800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:17.946978 29800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:17.956516 29807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:17.956566 29808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:17.956530 29810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:17.957087 29800 server_base.cc:1034] running on GCE node
I20260807 22:47:17.957628 29800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:17.958774 29800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:17.959942 29800 hybrid_clock.cc:648] HybridClock initialized: now 1786142837959917 us; error 37 us; skew 500 ppm
Aug 07 22:47:17 dist-test-slave-4tdj krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:17.965610 29800 init.cc:377] Logged in from keytab as kudu/127.26.189.254@KRBTEST.COM (short username kudu)
I20260807 22:47:17.966535 29800 webserver.cc:460] Webserver started at http://127.26.189.254:34227/ using document root <none> and password file <none>
I20260807 22:47:17.967065 29800 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:17.967144 29800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:17.967348 29800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:17.969340 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data/instance:
uuid: "cfcdf51ab25a45e293f9cc6484e935ea"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-4tdj"
server_key: "014d18189a160d562bf845b7feebb5f3"
server_key_iv: "5db90b8b44e7f86bd85bf590cb8dd2c5"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.969878 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal/instance:
uuid: "cfcdf51ab25a45e293f9cc6484e935ea"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-4tdj"
server_key: "014d18189a160d562bf845b7feebb5f3"
server_key_iv: "5db90b8b44e7f86bd85bf590cb8dd2c5"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.973408 29800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 22:47:17.975755 29821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:17.976876 29800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:47:17.976989 29800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal
uuid: "cfcdf51ab25a45e293f9cc6484e935ea"
format_stamp: "Formatted at 2026-08-07 22:47:17 on dist-test-slave-4tdj"
server_key: "014d18189a160d562bf845b7feebb5f3"
server_key_iv: "5db90b8b44e7f86bd85bf590cb8dd2c5"
server_key_version: "encryptionkey@0"
I20260807 22:47:17.977074 29800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:17.998636 29800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:18.001670 29800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:18.001848 29800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:18.008533 29800 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.254:36835
I20260807 22:47:18.008798 29901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.254:36835 every 8 connection(s)
I20260807 22:47:18.009444 29800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/data/info.pb
I20260807 22:47:18.010573 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29800
I20260807 22:47:18.010730 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-0/wal/instance
I20260807 22:47:18.011058 27383 external_mini_cluster.cc:1428] Setting key 2b673232b03c277c01d26f9dd4c19fd9
I20260807 22:47:18.012643 29902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:18.016479 29902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.189.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.253@KRBTEST.COM" created.
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:18.031028 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.018720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36835 (local address 127.0.0.1:39612)
0807 22:47:18.019242 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.019290 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.020112 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.020833 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.020838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.021153 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.021396 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.021409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.022651 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.022655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.023699 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.023711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.024386 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.024390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.024499 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.025036 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.025055 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.026622 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.028308 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.028316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.028325 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.028532 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.028824 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.028827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.028829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.028935 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.029456 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.029461 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.029585 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.029724 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.030101 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":98,"thread_start_us":148,"threads_started":1}
I20260807 22:47:18.031028 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.018718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.26.189.254:36835)
0807 22:47:18.019111 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.019129 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.019406 (+ 277us) server_negotiation.cc:407] Connection header received
0807 22:47:18.020412 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.020417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.020614 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.020806 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.021522 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.022546 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.023830 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.024225 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.024346 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.026749 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.026755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.026767 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.026791 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.028210 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.028634 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.028640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.028642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.028743 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.029014 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.029017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.029019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.029349 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.029466 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.029805 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.030040 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
I20260807 22:47:18.033315 29902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:18.034107 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.033775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45090)
0807 22:47:18.033912 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.034014 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 22:47:18.034346 29822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
I20260807 22:47:18.035176 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.034932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45104)
0807 22:47:18.035056 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.035121 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 22:47:18.035298 29902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:39483: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab.
Entry for principal kudu/127.26.189.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab.
I20260807 22:47:18.037190 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:39483
--webserver_interface=127.26.189.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab
--principal=kudu/127.26.189.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36835,127.26.189.253:39483,127.26.189.252:37805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:47:18.084528 29902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } attempt: 1
I20260807 22:47:18.085446 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.085064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45118)
0807 22:47:18.085256 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.085341 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 22:47:18.086174 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.085960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45126)
0807 22:47:18.086053 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.086109 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:47:18.086335 29902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:39483: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
W20260807 22:47:18.146097 29916 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:47:18.146390 29916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:18.146436 29916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:18.146493 29916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:18.150398 29916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:18.150449 29916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:18.150465 29916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:18.150482 29916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:18.150509 29916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:18.154482 29902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } attempt: 2
I20260807 22:47:18.155285 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.154978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45140)
0807 22:47:18.155131 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.155198 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:47:18.155929 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.155724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45142)
0807 22:47:18.155815 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.155866 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:47:18.155970 29916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36835,127.26.189.253:39483,127.26.189.252:37805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.253.keytab
--principal=kudu/127.26.189.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.253:39483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.189.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
W20260807 22:47:18.156047 29902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.253:39483: Network error: Client connection negotiation failed: client connection to 127.26.189.253:39483: connect: Connection refused (error 111)
I20260807 22:47:18.157001 29916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:18.157960 29916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:47:18.166031 29925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:18.166059 29924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:18.166280 29929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:18.166652 29916 server_base.cc:1034] running on GCE node
I20260807 22:47:18.167215 29916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:18.168294 29916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:18.169457 29916 hybrid_clock.cc:648] HybridClock initialized: now 1786142838169434 us; error 53 us; skew 500 ppm
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:18.172112 29916 init.cc:377] Logged in from keytab as kudu/127.26.189.253@KRBTEST.COM (short username kudu)
I20260807 22:47:18.172919 29916 webserver.cc:460] Webserver started at http://127.26.189.253:42929/ using document root <none> and password file <none>
I20260807 22:47:18.173410 29916 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:18.173465 29916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:18.173640 29916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:18.175614 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data/instance:
uuid: "d57f5a4de19349a592cfb646847ce51b"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-4tdj"
server_key: "e94a6b0c9a60f31be2ad99a8412d3eef"
server_key_iv: "8391cb6d02da1febf5a91b634f90cf74"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.176121 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal/instance:
uuid: "d57f5a4de19349a592cfb646847ce51b"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-4tdj"
server_key: "e94a6b0c9a60f31be2ad99a8412d3eef"
server_key_iv: "8391cb6d02da1febf5a91b634f90cf74"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.179528 29916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 22:47:18.181859 29937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:18.183037 29916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:47:18.183143 29916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal
uuid: "d57f5a4de19349a592cfb646847ce51b"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-4tdj"
server_key: "e94a6b0c9a60f31be2ad99a8412d3eef"
server_key_iv: "8391cb6d02da1febf5a91b634f90cf74"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.183235 29916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:18.199823 29916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:18.202466 29916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:18.202643 29916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:18.209362 29916 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.253:39483
I20260807 22:47:18.209620 30020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.253:39483 every 8 connection(s)
I20260807 22:47:18.210316 29916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/data/info.pb
I20260807 22:47:18.212929 30022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:18.213249 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 29916
I20260807 22:47:18.213363 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-1/wal/instance
I20260807 22:47:18.213654 27383 external_mini_cluster.cc:1428] Setting key c3604126b04ad931c887b3826b0714c5
I20260807 22:47:18.217096 30022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.189.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.189.252@KRBTEST.COM" created.
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:18.231535 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.219353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39628 (local address 127.26.189.254:36835)
0807 22:47:18.219529 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.219534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.220391 (+ 857us) server_negotiation.cc:407] Connection header received
0807 22:47:18.221606 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.221609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.221660 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.221738 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.223361 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.223894 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.224980 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.225242 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.225333 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.227970 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.227974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.227976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.228008 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.229482 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.229962 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.229966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.229968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.230069 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.230376 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.230380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.230382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.230613 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.230718 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.231232 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.231362 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:47:18.232223 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.219702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36835 (local address 127.0.0.1:39628)
0807 22:47:18.220206 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.220268 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.221333 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.222002 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.222011 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.222536 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.223208 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.223230 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.224010 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.224012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.224876 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.224885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.225373 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.225377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.225542 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.226149 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.226165 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.227775 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.229597 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.229604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.229616 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.229865 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.230148 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.230150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.230152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.230271 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.230720 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.230727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.230970 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.231130 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.231371 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":101,"threads_started":1}
I20260807 22:47:18.233608 30022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.189.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab.
Entry for principal kudu/127.26.189.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab.
I20260807 22:47:18.241196 27383 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
/tmp/dist-test-taskp7gIfB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:37805
--webserver_interface=127.26.189.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab
--principal=kudu/127.26.189.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.189.212:44815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.189.254:36835,127.26.189.253:39483,127.26.189.252:37805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:47:18.242009 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.234102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45154)
0807 22:47:18.234279 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.234312 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.234386 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.234886 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.234888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.235012 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.235104 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.235109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.236027 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.236030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.236612 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.236622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.237185 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.237188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.237258 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.237675 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.237688 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.238890 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.240673 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.240676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.240677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.240848 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.241130 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.241132 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.241134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.241196 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.241654 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.241656 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.241731 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.241810 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.241900 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 22:47:18.242215 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.234210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.26.189.253:39483)
0807 22:47:18.234499 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.234509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.234531 (+ 22us) server_negotiation.cc:407] Connection header received
0807 22:47:18.234574 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.234579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.234770 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.234909 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.235190 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.235901 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.236715 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.237090 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.237170 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.239014 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.239023 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.239034 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.239054 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.240569 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.240944 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.240950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.240952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.241040 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.241269 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.241272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.241274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.241555 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.241687 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.241907 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.242046 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":105,"threads_started":1}
I20260807 22:47:18.244135 30022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:18.244808 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.244556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50508)
0807 22:47:18.244671 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.244747 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 22:47:18.245005 29942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
I20260807 22:47:18.245821 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.245639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50514)
0807 22:47:18.245730 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.245773 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 22:47:18.245960 30022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:37805: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
I20260807 22:47:18.247177 29902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } attempt: 3
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:47:18.255189 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.247617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.26.189.253:39483)
0807 22:47:18.247706 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.247710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.247763 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:47:18.247885 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.247888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.247926 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.247986 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.248383 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.248878 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.249610 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.250262 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.250384 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.252431 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.252435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.252437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.252460 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.253701 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.254075 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.254079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.254081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.254150 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.254452 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.254455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.254457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.254625 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.254709 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.254954 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.255050 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:47:18.255195 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.247584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45168)
0807 22:47:18.247707 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.247721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.247809 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.248015 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.248018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.248183 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.248278 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.248283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.248975 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.248978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.249499 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.249507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.250386 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.250390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.250494 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.250885 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.250900 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.252304 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.253800 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.253803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.253805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.253988 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.254231 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.254234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.254236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.254368 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.254714 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.254717 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.254809 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.254893 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.255027 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 22:47:18.256042 29902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:18.256644 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.256436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50530)
0807 22:47:18.256538 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.256587 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 22:47:18.257270 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.257042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50542)
0807 22:47:18.257175 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.257219 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 22:47:18.257391 29902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:37805: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
I20260807 22:47:18.288533 29902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } attempt: 1
I20260807 22:47:18.289273 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.288995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50548)
0807 22:47:18.289136 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.289194 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 22:47:18.289983 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.289782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50554)
0807 22:47:18.289884 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.289927 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 22:47:18.290112 29902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:37805: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
I20260807 22:47:18.295185 30022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } attempt: 1
I20260807 22:47:18.295835 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.295602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50570)
0807 22:47:18.295720 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.295771 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 22:47:18.296504 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.296313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50576)
0807 22:47:18.296407 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.296449 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 22:47:18.296618 30022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:37805: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
W20260807 22:47:18.352499 30035 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:47:18.352756 30035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:47:18.352778 30035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:47:18.352835 30035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:47:18.356768 30035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:47:18.356822 30035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:47:18.356840 30035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:47:18.356855 30035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:47:18.356870 30035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:47:18.361941 30035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.189.212:44815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.189.254:36835,127.26.189.253:39483,127.26.189.252:37805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/krb5kdc/kudu_127.26.189.252.keytab
--principal=kudu/127.26.189.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.189.252:37805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.189.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 22:03:03 UTC on ccycloud.quasar-mvmwel.root.comops.site
build id 4573647
I20260807 22:47:18.362952 30035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:47:18.363914 30035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:47:18.364760 30022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } attempt: 2
I20260807 22:47:18.365249 29902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } attempt: 2
I20260807 22:47:18.365634 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.365274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50586)
0807 22:47:18.365436 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.365555 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 22:47:18.365983 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.365654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50592)
0807 22:47:18.365832 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.365887 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 22:47:18.366636 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.366403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50606)
0807 22:47:18.366519 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.366571 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:47:18.366796 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.366540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50614)
0807 22:47:18.366664 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.366720 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 22:47:18.366992 29902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:37805: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
W20260807 22:47:18.366979 30022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.189.252:37805: Network error: Client connection negotiation failed: client connection to 127.26.189.252:37805: connect: Connection refused (error 111)
W20260807 22:47:18.372350 30047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:18.372349 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:47:18.372385 30048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:47:18.372749 30035 server_base.cc:1034] running on GCE node
I20260807 22:47:18.373266 30035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:47:18.374269 30035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:47:18.375451 30035 hybrid_clock.cc:648] HybridClock initialized: now 1786142838375436 us; error 29 us; skew 500 ppm
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:47:18.378010 30035 init.cc:377] Logged in from keytab as kudu/127.26.189.252@KRBTEST.COM (short username kudu)
I20260807 22:47:18.378854 30035 webserver.cc:460] Webserver started at http://127.26.189.252:45103/ using document root <none> and password file <none>
I20260807 22:47:18.379359 30035 fs_manager.cc:359] Metadata directory not provided
I20260807 22:47:18.379422 30035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:47:18.379621 30035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:47:18.381623 30035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data/instance:
uuid: "f60944bc95124811883232459c2b6610"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-4tdj"
server_key: "a8b36e7c4a88fd945dedbb071c803b93"
server_key_iv: "a928dad83515eaf13d6630549ea17f0d"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.382160 30035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal/instance:
uuid: "f60944bc95124811883232459c2b6610"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-4tdj"
server_key: "a8b36e7c4a88fd945dedbb071c803b93"
server_key_iv: "a928dad83515eaf13d6630549ea17f0d"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.385663 30035 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:47:18.388088 30066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:47:18.389245 30035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 22:47:18.389353 30035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data,/tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal
uuid: "f60944bc95124811883232459c2b6610"
format_stamp: "Formatted at 2026-08-07 22:47:18 on dist-test-slave-4tdj"
server_key: "a8b36e7c4a88fd945dedbb071c803b93"
server_key_iv: "a928dad83515eaf13d6630549ea17f0d"
server_key_version: "encryptionkey@0"
I20260807 22:47:18.389441 30035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:47:18.422043 30035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:47:18.424821 30035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:47:18.424999 30035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:47:18.431778 30035 rpc_server.cc:307] RPC server started. Bound to: 127.26.189.252:37805
I20260807 22:47:18.432034 30145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.189.252:37805 every 8 connection(s)
I20260807 22:47:18.432720 30035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/data/info.pb
I20260807 22:47:18.435397 30147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:47:18.438563 27383 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu as pid 30035
I20260807 22:47:18.438714 27383 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp7gIfB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142556986508-27383-0/minicluster-data/master-2/wal/instance
I20260807 22:47:18.439045 27383 external_mini_cluster.cc:1428] Setting key 8299445660a2d7be77c7912d36aa11b9
I20260807 22:47:18.439254 30147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:47:18.451939 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.441167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.26.189.254:36835)
0807 22:47:18.441312 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.441317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.441709 (+ 392us) server_negotiation.cc:407] Connection header received
0807 22:47:18.442041 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.442044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.442100 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.442177 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.443196 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.443972 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.445394 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.445759 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.445867 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.448448 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.448452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.448454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.448483 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.449962 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.450482 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.450486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.450488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.450573 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.450889 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.450893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.450894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.451075 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.451160 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.451657 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.451777 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:47:18.452687 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.441089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.254:36835 (local address 127.0.0.1:39632)
0807 22:47:18.441545 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.441606 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.441870 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.442245 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.442251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.442710 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.443056 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.443071 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.444115 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.444118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.445261 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.445273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.445914 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.445917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.446034 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.446641 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.446662 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.448298 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.450072 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.450081 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.450091 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.450384 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.450660 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.450663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.450665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.450794 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.451176 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.451182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.451415 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.451561 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.451852 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":21,"spinlock_wait_cycles":768,"thread_start_us":117,"threads_started":1}
I20260807 22:47:18.452689 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.440422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.26.189.252:37805)
0807 22:47:18.440819 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.440833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.440866 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:47:18.441577 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.441596 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.441896 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.442076 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.443008 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.444208 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.445394 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.446003 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.446079 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.448893 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.448898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.448907 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.448928 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.450355 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.450745 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.450749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.450751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.450822 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.451069 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.451073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.451074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.451408 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.451593 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.451789 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.451913 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":265,"spinlock_wait_cycles":4864,"thread_start_us":107,"threads_started":1}
I20260807 22:47:18.453874 30147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:47:18.458139 30022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } attempt: 3
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.254@KRBTEST.COM
I20260807 22:47:18.461817 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.454450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.26.189.253:39483)
0807 22:47:18.454572 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.454576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.454589 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:47:18.454666 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.454668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.454706 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.454780 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.455166 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.455636 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.456393 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.456647 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.456720 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.458598 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.458602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.458604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.458629 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.460149 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.460529 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.460533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.460535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.460623 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.460908 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.460912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.460914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.461099 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.461167 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.461555 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.461659 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:47:18.462257 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.454348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.253:39483 (local address 127.0.0.1:45176)
0807 22:47:18.454503 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.454518 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.454586 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.454810 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.454812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.454966 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.455075 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.455082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.455734 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.455737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.456284 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.456292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.456746 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.456750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.456847 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.457211 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.457224 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.458407 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.460222 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.460225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.460227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.460442 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.460704 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.460706 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.460708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.460795 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.461243 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.461245 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.461335 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.461990 (+ 655us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.462118 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.253@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:47:18.462909 30147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:47:18.464005 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.454960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.26.189.254:36835)
0807 22:47:18.455059 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.455062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.455075 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:47:18.455229 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.455233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.455273 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.455334 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.455898 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.456332 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.457173 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.457375 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.457446 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.460125 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.460129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.460132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.460156 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.461852 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.462512 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.462515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.462516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.462591 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.462858 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.462861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.462862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.463098 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.463302 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.463739 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.463854 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:47:18.466126 29902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } attempt: 3
I20260807 22:47:18.467185 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.458534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50630)
0807 22:47:18.458692 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.458704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.458771 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.459163 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.459167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.459318 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.459410 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.459416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.460134 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.460138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.460803 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.460813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.461365 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.461368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.461458 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.461810 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.461823 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.463291 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.465444 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.465448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.465449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.465684 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.465950 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.465954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.465955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.466048 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.466477 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.466480 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.466575 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.466651 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.466775 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 22:47:18.467752 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.458894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.26.189.252:37805)
0807 22:47:18.459000 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.459004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.459014 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:47:18.459038 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.459040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.459069 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.459129 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.459545 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.460042 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.460912 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.461262 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.461351 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.463399 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.463402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.463405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.463427 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.465347 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.465757 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.465760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.465762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.465836 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.466142 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.466145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.466146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.466369 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.467441 (+ 1072us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.467509 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.467618 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142838, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.252@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.189.253@KRBTEST.COM
I20260807 22:47:18.471051 30022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: Bootstrap starting.
I20260807 22:47:18.472841 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.463329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50642)
0807 22:47:18.463439 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.463451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.463523 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.464168 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.464170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.464317 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.464401 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.464408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.465901 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.465904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.466595 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.466604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.467153 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.467157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.467231 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.467619 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.467631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.468872 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.471514 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.471517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.471519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.471727 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.471972 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.471975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.471977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.472058 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.472458 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.472461 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.472533 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.472606 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.472714 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
Aug 07 22:47:18 dist-test-slave-4tdj krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142837, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.189.254@KRBTEST.COM for kudu/127.26.189.252@KRBTEST.COM
I20260807 22:47:18.473384 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.463479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.26.189.252:37805)
0807 22:47:18.463953 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.463964 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.463976 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:47:18.464006 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.464009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.464054 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.464512 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.464586 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.465805 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.466701 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.467032 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.467397 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.468997 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.469000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.469002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.469023 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.471414 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.471823 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.471825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.471827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.471896 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.472144 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.472147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.472149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.472363 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.473072 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.473141 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.473242 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":114,"threads_started":1}
I20260807 22:47:18.474068 30022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:18.474783 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.465125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45186 (local address 127.26.189.253:39483)
0807 22:47:18.465241 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.465245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.465595 (+ 350us) server_negotiation.cc:407] Connection header received
0807 22:47:18.465685 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.465688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.465727 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.465794 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.466801 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.467411 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.468476 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.468763 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.468835 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.471490 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.471493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.471496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.471519 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.472933 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.473418 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.473421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.473423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.473502 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.473800 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.473803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.473805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.474010 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.474155 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.474564 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.474660 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:47:18.474824 30022 log.cc:824] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:18.476187 30147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: Bootstrap starting.
I20260807 22:47:18.476949 30022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: No bootstrap required, opened a new log
I20260807 22:47:18.478276 27383 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:47:18.478638 30147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:18.479435 30147 log.cc:824] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:18.480171 30022 raft_consensus.cc:348] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.480441 30022 raft_consensus.cc:729] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d57f5a4de19349a592cfb646847ce51b, State: Initialized, Role: FOLLOWER
I20260807 22:47:18.479589 29864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cfcdf51ab25a45e293f9cc6484e935ea"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39636
I20260807 22:47:18.480769 30022 consensus_queue.cc:260] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.481205 30161 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.481328 30161 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.481540 30147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: No bootstrap required, opened a new log
I20260807 22:47:18.482069 30022 sys_catalog.cc:564] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:18.485608 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.466494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.189.252:37805 (local address 127.0.0.1:50648)
0807 22:47:18.466652 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:47:18.466665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:47:18.466730 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:47:18.468074 (+ 1344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:47:18.468077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:47:18.468241 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:47:18.468329 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.468336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.469408 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.469412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.470139 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:47:18.470151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.471387 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.471390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:47:18.471486 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.472049 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:47:18.472063 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:47:18.473913 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:47:18.483947 (+ 10034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.483950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.483952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.484203 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.484520 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:47:18.484523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:47:18.484525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:47:18.484627 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:47:18.485141 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:47:18.485144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:47:18.485253 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 22:47:18.485335 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:47:18.485458 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 22:47:18.486331 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.466916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.26.189.252:37805)
0807 22:47:18.467223 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.467670 (+ 447us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.467686 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:47:18.467721 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.467724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.467767 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.468436 (+ 669us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.468564 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.469300 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.470264 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.471252 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.479939 (+ 8687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.480040 (+ 101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.480043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.480046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.480062 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.483821 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.484330 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.484334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.484336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.484426 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.484755 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.484758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.484760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.485029 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.485855 (+ 826us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.485928 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.486033 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
W20260807 22:47:18.487597 30177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:18.487665 30177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:18.485584 29916 master_runner.cc:186] Error getting master registration for 127.26.189.254:36835: OK, instance_id { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" instance_seqno: 1786142838006762 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:47:18.488795 29916 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:47:18.489223 29902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea: Bootstrap starting.
I20260807 22:47:18.489152 30147 raft_consensus.cc:348] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.489387 30147 raft_consensus.cc:729] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f60944bc95124811883232459c2b6610, State: Initialized, Role: FOLLOWER
I20260807 22:47:18.489727 30147 consensus_queue.cc:260] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.490159 30180 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.490264 30180 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.490971 30147 sys_catalog.cc:564] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:18.491765 29902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea: Neither blocks nor log segments found. Creating new log.
I20260807 22:47:18.492619 29902 log.cc:824] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea: Log is configured to *not* fsync() on all Append() calls
I20260807 22:47:18.494666 29902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea: No bootstrap required, opened a new log
W20260807 22:47:18.495947 30199 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:47:18.496073 30199 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:47:18.494455 30035 master_runner.cc:186] Error getting master registration for 127.26.189.254:36835: OK, instance_id { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" instance_seqno: 1786142838006762 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:47:18.497450 29902 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.497627 30035 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:47:18.497666 29902 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfcdf51ab25a45e293f9cc6484e935ea, State: Initialized, Role: FOLLOWER
I20260807 22:47:18.497974 29902 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.498531 30201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.498795 30201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.499207 29902 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:47:18.499290 29864 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:47:18.499399 29864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:18.500559 29864 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.501492 29864 leader_election.cc:288] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [CANDIDATE]: Term 1 election: Requested vote from peers d57f5a4de19349a592cfb646847ce51b (127.26.189.253:39483), f60944bc95124811883232459c2b6610 (127.26.189.252:37805)
I20260807 22:47:18.502015 30101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f60944bc95124811883232459c2b6610"
I20260807 22:47:18.502223 30101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:18.502238 29978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d57f5a4de19349a592cfb646847ce51b"
I20260807 22:47:18.502594 29978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:47:18.503885 29978 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfcdf51ab25a45e293f9cc6484e935ea in term 1.
I20260807 22:47:18.504011 30101 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfcdf51ab25a45e293f9cc6484e935ea in term 1.
W20260807 22:47:18.504194 29832 tablet.cc:2369] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:47:18.504752 29827 leader_election.cc:302] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfcdf51ab25a45e293f9cc6484e935ea, f60944bc95124811883232459c2b6610; no voters:
I20260807 22:47:18.505865 30201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:47:18.506139 30201 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [term 1 LEADER]: Becoming Leader. State: Replica: cfcdf51ab25a45e293f9cc6484e935ea, State: Running, Role: LEADER
I20260807 22:47:18.506523 30201 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } }
I20260807 22:47:18.508879 30201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfcdf51ab25a45e293f9cc6484e935ea. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.509042 30201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:18.509271 29800 master_runner.cc:263] No leader master found from master cfcdf51ab25a45e293f9cc6484e935ea
I20260807 22:47:18.509338 29800 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:47:18.509503 30236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:47:18.513441 30236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:47:18.523646 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.510678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.26.189.254:36835)
0807 22:47:18.510784 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.510788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.514579 (+ 3791us) server_negotiation.cc:407] Connection header received
0807 22:47:18.514997 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.515000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.515046 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.515094 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.516024 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.516801 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.517961 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.518308 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.518379 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.520154 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.520158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.520160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.520186 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.521781 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.522262 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.522265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.522267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.522381 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.522688 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.522691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.522693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.522903 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.523010 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.523358 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.523471 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:47:18.530279 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.511157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50664 (local address 127.26.189.252:37805)
0807 22:47:18.511269 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.511272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.514854 (+ 3582us) server_negotiation.cc:407] Connection header received
0807 22:47:18.514949 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.514952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.515014 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.515093 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.519197 (+ 4104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.519979 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.524028 (+ 4049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.524373 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.524450 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.526218 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.526222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.526225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.526252 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.528442 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.528980 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.528984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.528985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.529073 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.529353 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.529356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.529358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.529574 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.529689 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.530000 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.530122 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:47:18.534725 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:18.511549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.26.189.253:39483)
0807 22:47:18.511647 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.511651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.517362 (+ 5711us) server_negotiation.cc:407] Connection header received
0807 22:47:18.518188 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.518191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.518239 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.518287 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.521109 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.521861 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.524933 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:18.525298 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:18.525374 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:18.530683 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:47:18.530687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:47:18.530690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:47:18.530718 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:47:18.532948 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.533411 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.533415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.533417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.533500 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:47:18.533790 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:47:18.533793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:47:18.533796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:47:18.533992 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:47:18.534106 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:18.534446 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:47:18.534573 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:47:18.536702 29978 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [term 1 FOLLOWER]: Refusing update from remote peer cfcdf51ab25a45e293f9cc6484e935ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:18.536705 30101 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [term 1 FOLLOWER]: Refusing update from remote peer cfcdf51ab25a45e293f9cc6484e935ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:47:18.537355 30231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:18.537520 30201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:47:18.541692 30161 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfcdf51ab25a45e293f9cc6484e935ea. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.541797 30161 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.542346 30180 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfcdf51ab25a45e293f9cc6484e935ea. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.542449 30180 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.559314 30201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.559461 30201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:18.559567 30242 mvcc.cc:205] Tried to move back new op lower bound from 7316041066594103296 to 7316041066526826496. Current Snapshot: MvccSnapshot[applied={T|T < 7316041066594103296}]
I20260807 22:47:18.560853 30180 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.560977 30180 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.561659 30247 mvcc.cc:205] Tried to move back new op lower bound from 7316041066594103296 to 7316041066526826496. Current Snapshot: MvccSnapshot[applied={T|T < 7316041066594103296}]
I20260807 22:47:18.568527 30231 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.568676 30231 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea [sys.catalog]: This master's current role is: LEADER
I20260807 22:47:18.569696 30161 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfcdf51ab25a45e293f9cc6484e935ea" member_type: VOTER last_known_addr { host: "127.26.189.254" port: 36835 } } peers { permanent_uuid: "d57f5a4de19349a592cfb646847ce51b" member_type: VOTER last_known_addr { host: "127.26.189.253" port: 39483 } } peers { permanent_uuid: "f60944bc95124811883232459c2b6610" member_type: VOTER last_known_addr { host: "127.26.189.252" port: 37805 } } }
I20260807 22:47:18.569778 30246 mvcc.cc:205] Tried to move back new op lower bound from 7316041066594103296 to 7316041066526826496. Current Snapshot: MvccSnapshot[applied={T|T < 7316041066594103296}]
I20260807 22:47:18.569818 30161 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:47:18.573312 30236 catalog_manager.cc:1347] Generated new cluster ID: 7d869246f94646c9a7a2845e0b03cd0d
I20260807 22:47:18.573385 30236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:47:18.637820 30236 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:47:18.639232 30236 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:47:18.669257 30236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfcdf51ab25a45e293f9cc6484e935ea: Generated new TSK 0
I20260807 22:47:18.670130 30236 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:47:18.703220 29905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.701498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39652 (local address 127.26.189.254:36835)
0807 22:47:18.701632 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.701637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.702165 (+ 528us) server_negotiation.cc:407] Connection header received
0807 22:47:18.702342 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.702346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.702399 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.702479 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.703082 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39652: BlockingRecv error: recv got EOF from 127.0.0.1:39652 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 22:47:18.703225 30156 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.701580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.26.189.252:37805)
0807 22:47:18.701703 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.701707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.702243 (+ 536us) server_negotiation.cc:407] Connection header received
0807 22:47:18.702434 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.702438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.702488 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.702540 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.703104 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50670: BlockingRecv error: recv got EOF from 127.0.0.1:50670 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 22:47:18.703369 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.701591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.26.189.253:39483)
0807 22:47:18.701701 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.701706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.702482 (+ 776us) server_negotiation.cc:407] Connection header received
0807 22:47:18.702654 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.702657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.702706 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.702756 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.703260 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45206: BlockingRecv error: recv got EOF from 127.0.0.1:45206 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 22:47:18.706369 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.705402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.26.189.253:39483)
0807 22:47:18.705494 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.705498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.705623 (+ 125us) server_negotiation.cc:407] Connection header received
0807 22:47:18.705742 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.705744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.705810 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.705888 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.706255 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45214: BlockingRecv error: recv got EOF from 127.0.0.1:45214 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 22:47:18.709919 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.709176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.26.189.253:39483)
0807 22:47:18.709285 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.709288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.709298 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:47:18.709333 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.709335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.709369 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.709433 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.709836 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45224: BlockingRecv error: recv got EOF from 127.0.0.1:45224 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 22:47:18.714637 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.713591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.26.189.253:39483)
0807 22:47:18.713701 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.713705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.713914 (+ 209us) server_negotiation.cc:407] Connection header received
0807 22:47:18.714000 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.714003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.714046 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.714122 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.714553 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45230: BlockingRecv error: recv got EOF from 127.0.0.1:45230 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 22:47:18.721335 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.720322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.26.189.253:39483)
0807 22:47:18.720470 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.720474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.720575 (+ 101us) server_negotiation.cc:407] Connection header received
0807 22:47:18.720738 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.720741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.720786 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.720850 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.721241 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45232: BlockingRecv error: recv got EOF from 127.0.0.1:45232 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 22:47:18.731875 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.730968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.26.189.253:39483)
0807 22:47:18.731112 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.731115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.731197 (+ 82us) server_negotiation.cc:407] Connection header received
0807 22:47:18.731293 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.731295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.731335 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.731400 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.731770 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45246: BlockingRecv error: recv got EOF from 127.0.0.1:45246 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 22:47:18.750557 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.749535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45256 (local address 127.26.189.253:39483)
0807 22:47:18.749672 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.749676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.749793 (+ 117us) server_negotiation.cc:407] Connection header received
0807 22:47:18.749902 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.749905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.749948 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.750012 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.750437 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45256: BlockingRecv error: recv got EOF from 127.0.0.1:45256 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 22:47:18.785619 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.784377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.26.189.253:39483)
0807 22:47:18.784506 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.784512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.784748 (+ 236us) server_negotiation.cc:407] Connection header received
0807 22:47:18.784892 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.784896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.784944 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.785013 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.785499 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45262: BlockingRecv error: recv got EOF from 127.0.0.1:45262 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 22:47:18.852783 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.851817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.26.189.253:39483)
0807 22:47:18.851978 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.851981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.852064 (+ 83us) server_negotiation.cc:407] Connection header received
0807 22:47:18.852202 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.852204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.852252 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.852309 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.852660 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45264: BlockingRecv error: recv got EOF from 127.0.0.1:45264 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 22:47:18.984157 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:18.982940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.26.189.253:39483)
0807 22:47:18.983073 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:18.983078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:18.983313 (+ 235us) server_negotiation.cc:407] Connection header received
0807 22:47:18.983432 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:18.983434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:18.983486 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:18.983566 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:18.984036 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45270: BlockingRecv error: recv got EOF from 127.0.0.1:45270 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 22:47:19.243588 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:47:19.242451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.26.189.253:39483)
0807 22:47:19.242605 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.242611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.242691 (+ 80us) server_negotiation.cc:407] Connection header received
0807 22:47:19.242852 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.242855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.242903 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.242974 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:47:19.243458 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45286: BlockingRecv error: recv got EOF from 127.0.0.1:45286 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 22:47:19.488951 30177 catalog_manager.cc:1259] Loaded cluster ID: 7d869246f94646c9a7a2845e0b03cd0d
I20260807 22:47:19.489048 30177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: loading cluster ID for follower catalog manager: success
I20260807 22:47:19.490952 30177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: acquiring CA information for follower catalog manager: success
I20260807 22:47:19.492167 30177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d57f5a4de19349a592cfb646847ce51b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:47:19.497125 30199 catalog_manager.cc:1259] Loaded cluster ID: 7d869246f94646c9a7a2845e0b03cd0d
I20260807 22:47:19.497180 30199 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: loading cluster ID for follower catalog manager: success
I20260807 22:47:19.498675 30199 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: acquiring CA information for follower catalog manager: success
I20260807 22:47:19.499707 30199 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f60944bc95124811883232459c2b6610: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:47:19.760977 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.757506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.26.189.253:39483)
0807 22:47:19.757723 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.757727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.757806 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:47:19.757922 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.757924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.758051 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.758161 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:19.758399 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.758932 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.759707 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.759927 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.759999 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.760233 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:19.760656 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:19.760743 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.760769 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.760817 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260807 22:47:19.767287 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.763750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.26.189.252:37805)
0807 22:47:19.763984 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.763988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.764000 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:47:19.764080 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.764082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.764191 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.764279 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:19.764572 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.765078 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.765942 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.766186 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.766263 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.766599 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:19.766954 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:19.767048 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.767125 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.767171 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260807 22:47:19.773926 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.769912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.26.189.252:37805)
0807 22:47:19.770010 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.770014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.770501 (+ 487us) server_negotiation.cc:407] Connection header received
0807 22:47:19.770660 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.770663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.770712 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.770780 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:19.771108 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.771844 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.772831 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.773105 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.773202 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.773520 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:19.773584 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:19.773641 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.773746 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.773795 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:47:19.773986 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.770109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.26.189.253:39483)
0807 22:47:19.770214 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.770218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.770630 (+ 412us) server_negotiation.cc:407] Connection header received
0807 22:47:19.770678 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.770680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.770722 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.770780 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:19.771188 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.771933 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.772776 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.773079 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.773133 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.773541 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:19.773606 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:19.773648 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.773813 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.773860 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:47:19.774179 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:47:19.769814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39658 (local address 127.26.189.254:36835)
0807 22:47:19.770345 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 22:47:19.770349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:47:19.770365 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:47:19.770458 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:47:19.770462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:47:19.770630 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:47:19.770736 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:47:19.770929 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.771540 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.772564 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:47:19.772885 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:47:19.772976 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:47:19.773273 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:47:19.773595 (+ 322us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:47:19.773674 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:47:19.773985 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 22:47:19.774047 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":65,"threads_started":1}
I20260807 22:47:19.777133 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29800
I20260807 22:47:19.788511 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 29916
I20260807 22:47:19.799327 27383 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp7gIfB/build/debug/bin/kudu with pid 30035
2026-08-07T22:47:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3238 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6403 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (282815 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST